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PREFACE 


a le oe of learning which make use of conditioning 
principles are not related in a simple manner to the 
facts from conditioning experiments. This book represents an 
effort to place the facts and theories into some sort of order 
through critical exposition. The net result is not a finished 
theory; in many instances we have found it necessary to 
point to several alternative conceptions. The examination of 
the relation of conditioning to other basic learning experi- 
ments has at other points revealed serious gaps in our knowl- 
edge which future experimentation must fill. 

Although conditioning has sometimes been proposed as a 
basic concept for the fields of cerebral physiology, of mental 
hygiene and personality, and of the higher thought processes, 
we have found it desirable to place greatest emphasis upon 
the relation of conditioning to learning theory. In the later 
chapters we have surveyed the possibilities of applying con- 
ditioning concepts within some of the other fields. 

Our own experimental work in conditioning was first un- 
dertaken in the laboratories of Professors Raymond Dodge, 
Harold S. Burr, and John F. Fulton of Yale University, and 
to these and other teachers, colleagues, and collaborators, we 
owe much. We have profited greatly from the reading and 
criticism of the entire manuscript by Professor R. M. Elliott, 
editor of this series. The manuscript was improved through 
suggestions following critical reading of different portions by 
Re eeinench) Ne idali"G I. Peaken WV. d= blerons 
C. I. Hovland, C. L. Hull, J. L. Kennedy, A. A. Lumsdaine, 
T. L. McCulloch, N. E. Miller, Helen Peak, T. C. Ruch, 
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R. R. Sears, B. F. Skinner, K. W. Spence, E. C. Tolman, and 
S. B. Williams. That their suggestions were all useful to us 
does not mean that they have approved the use which we 
made of them. The kindness of Dr. G. H. S. Razran in fur- 
nishing us an advance copy of his bibliography is gratefully 
acknowledged. He and several other authors kindly placed 
in our hands manuscripts prior to publication. Material has 
been drawn upon from personal communications received 
from the following: W. H. Burnham, W. H. Gantt, 
K. S. Lashley, R. B. Loucks, Florence Mateer, J. J. B. Mor- 
gan, Richard Parmenter, J. B. Watson, and R. M. Yerkes. 

Acknowledgment is made to D. Appleton-Century Co., 
The Macmillan Company, Oxford University Press, and 
Prentice-Hall, Inc., from whose publications figures or quo- 
tations are reproduced, and to the editors and publishers of 
the following journals for similar reproductions: American 
Journal of Psychology, Archives of Psychology, Journal of 
Comparative Psychology, Journal of Experimental Psychol- 
ogy, Journal of General Psychology, Journal of Genetic Psy- 
chology, Proceedings of the National Academy of Sciences, 
Psychological Monographs, Psychological Review, and Psy- 


chosomatic Medicine. Citation to the original source is given 
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THE PLACE OF CONDITIONING IN PSYCHOLOGY 


HE CLASSICAL conditioning experiment is known to ev- 

ery student of physiology and psychology. When meat 
powder is placed in a dog’s mouth, saliva flows. This natural 
and relatively automatic behavior is known as an uncondi- 
tioned reflex. An incidental stimulus, such as the sound of a 
ringing bell, does not at first result in the flow of saliva. If 
the bell sounds each time just before the food is presented, 
however, it will eventually elicit salivation. The new response 
to the bell is known as a conditioned response, and the bell 
is called a conditioned stimulus. A relationship has been es- 
tablished between the bell and food, so that a response origi- 
nally made to the food is now made to the bell. 

This simple experiment, with its superstructure of theory, 
has had a profound influence within the fields of physiology, 
psychology, and mental hygiene. Despite the fact that it pro- 
vided a new technique for the investigation of many impor- 
tant problems, the influence of conditioning on experimental 
study was preceded and greatly overshadowed by its influ- 
ence on broader problems of theory. For many persons, con- 
ditioning seemed to promise a completely objective science of 
the so-called higher mental functions. Biology and physiol- 
ogy, which had made great strides in the study of simple re- 
flex activity, had been unable to use the same methods in 
the investigation of the complex functions of the brain. Ex- 
cept for a few experiments in localization, the study of higher 
processes had remained in the province of psychology, and 
physiology textbooks were forced to discuss cortical func- 
tions in mentalistic terms such as volition, attention, and 
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memory. Conditioning, it was now declared, provided a 
method for the investigation of the higher brain functions 
which was just as physiological as the methods for studying 
spinal reflexes. Psychology too wanted to become an objec- 
tive science like other sciences, and conditioning was offered 
as evidence of advance in this direction. The conditioned re- 
sponse was called the unit of habit by psychologists to whom 
habit was the most important concept in psychology; condi- 
tioning method was proclaimed as a substitute for introspec- 
tion, once the chief method of psychology; the conditioning 
process, said a distinguished psychologist-philosopher, brings 
mind into being (Holt, 1931, p. 28). 

Today uncritical enthusiasm for conditioning has de- 
creased sufficiently to make a sober historical estimate pos- 
sible. For this appraisal, it is essential to reconstruct the sci- 
entific temper at the time of the great ardor for conditioning 
and so reveal the reasons for its acceptance. The data of ex- 
perimental psychology gathered by other methods in the 
fields of both perception and learning have always been far 
richer in content than those gathered by the conditioning 
method. Because this was especially true during the time that 
conditioning was at the height of popularity, an interesting 
problem confronts the historian, Why was a new method 
hailed as a sign of the birth of an objective science of psy- 
chology before that method had been thoroughly tried out? 
The answer to this question will place conditioning in proper 
perspective and aid us in assigning it a rôle in psychology 
more nearly in proportion to its potential service. 


Preparation within Psychology for the Acceptance 
of Conditioning 


When new concepts are incorporated into a scientific sys- 
tem prior to the experimental demonstration of their ade- 
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quacy, it is apparent that there has arisen within the system 
some kind of preparation or readiness for their assimilation. 
When such preparation is lacking, important discoveries and 
new methods are often refused admittance. The time must 
somehow have been ripe for conditioning. Otherwise it would 
not have been proposed as a method for measuring thresholds 
until it had been compared with the highly refined threshold 
methods already available, nor as a basis for learning theory 
before some effort had been made to compare what it ac- 
complished with well-established findings from other types of 
experiments using problem-boxes, mazes, skilled tasks, and 
nonsense syllables. It was accepted, however, without such 
tests. 

The fundamental facts of conditioning were known for 
many years before their exploitation took place. Twitmyer 
had independently discovered the phenomenon in an Ameri- 
can psychological laboratory in 1902. Pavlov had published 
reports in English in 1906; Yerkes and Morgulis had re- 
viewed his work and recommended his method to animal 
psychologists in 1909. In Watson’s first book championing 
behaviorism (1914) the methods of Pavlov are discussed as 
somewhat inferior to other methods. It cannot be said that 
the conditioned response won its way into psychology be- 
cause it was hailed immediately as a great discovery, or be- 
cause psychologists were greatly impressed by the new 
methodology. 

Three factors in the history of psychology are important 
in understanding the emphasis which was subsequently given 
to conditioning in psychological theory. These were, first, the 
tradition of association, second, the growing importance of 
habit as a psychological category, and third, the trend toward 
objectivity. 

Association. Modern experimental psychology developed 
under the influence of association theory, inherited (with 
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modifications) from English empiricism. Emphasis upon the 
experiential basis of ideas had led to the formulation of 
laws of association to account for the learned relations be- 
tween ideas. The set of laws as conventionally stated in- 
cluded three primary ones (contiguity, similarity, and con- 
trast) and numerous secondary or quantitative laws which 
regulated the effectiveness of the primary ones. These laws, 
especially the secondary ones, are the antecedents of laws of 
learning formulated by Thorndike and others. The laws of 
association underwent many changes, and no one list was 
ever officially canonized. There was a noticeable tendency, 
however, to subjugate the laws of similarity and contrast to 
that of contiguity. The law of contiguity states that an asso- 
ciation is formed between experiences that occur close to- 
gether in space or in time. It is the one law which was found 
in all the lists. 

In an effort to find a physiological basis for association, 
the principle of contiguous excitation was proposed. As ex- 
pressed by William James the principle reads: 


When two elementary brain-processes have been active to- 
gether or in immediate succession, one of them, on reoccurring, 


tends to propagate its excitement into the other. (W 
1890, I, p. 566.) (W. James, 


With association reduced to contiguous events and stated as 
a relation between brain states, the transition to a physiologi- 
cal doctrine of conditioning is straightforward and does no 
violence to established modes of thought. 

Habit. Although psychology was prepared by association 
theory both for the elementarism of the conditioned response 
and for its brain physiology, association theory had placed 
no emphasis on habit and learning as important psychologi- 
cal concepts. Titchener, writing in 1909, preferred not to use 
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the word ‘learning’ in his chapter on association, excusing him- 
self on the grounds that learning was such a complex affair. 
In the following chapter (on memory and imagination) he 
devoted two pages to a discussion of slow and rapid learners, 
and to cramming. The word ‘habit’ was mentioned once in 
relation to the habit of introspection, and again in the chap- 
ter on attention. That is all. While Titchener was in some 
sense an associationist, it is evident from his writings that 
affiliation with association is not enough to force habit and 
learning into the foreground of a psychological system. 

It was another development, Darwinian biology, which 
brought habit into focus as a psychological concept. An or- 
ganism’s struggle to adapt to its environment involves muscu- 
lar movements of locomotion and the like, which resist 
description in purely conscious terms. Indeed, one character- 
istic of habitual acts appears to be that the more skillful 
they become, the less their details are represented in con- 
sciousness. The concept of habit breaks over the boundaries 
of an association theory limited to conscious processes. 

William James gave prominence to habit in his Principles, 
not only by writing an important and readable chapter, but 
by placing it early in his book (Vol. I, Chapter 4) among 
other chapters dealing with the fundamental characteristics 
of psychological and brain processes. The usual introductory 
discussions of sensation and perception he postponed until 
later chapters. James R. Angell at the University of Chicago 
gave habit a central place in his systematic psychology which 
came to be known as functionalism. Earlier association 
theories had been essentially physiological; in functionalism, 
association was given a biological flavor. While conscious- 
ness remained the primary subject-matter of psychology, 
investigation of adaptive behavior could not be confined to 
introspective studies. Animals came to be used increasingly 
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in studies of habit. Introspection in animal experiments was 
impossible; yet the experiments were producing significant 
information regarding the learning process. 

Objectivity. The study of habit contributed to a general 
trend in psychology away from so-called subjective methods 
to more objective types of observation. When psychology be- 
came a laboratory science it was on the road to objectivity. 
Laboratories meant control and measurement, and the for- 
mulation of problems amenable to study in quantitative terms. 
From biology came the voices of the mechanists calling for 
an objective study of life sciences, without recourse to men- 
talistic concepts. The famous paper by Beer, Bethe, and v. 
Uexküll (1899) suggested new terms for the usual psy- 
chological ones, substituting, for example, reception for sen- 
sation. Jacques Loeb, the leader of the movement, who re- 
sided in America from 1891 until his death, taught for some 
years at the University of Chicago, where the influence of 
biology on psychology was strong. 

Not without influence was a movement among philoso- 
phers to dispense with both dualism and subjective idealism, 
and to give a more objective account of mind. James enter- 
tained doubts about the need for a concept of consciousness 
in psychology, and Holt (1914) attempted a physiological 
account of the phenomena associated with the term. The 
movement was not confined to Harvard philosophers like 
James, Holt, and Perry. At Pennsylvania, for example, 
Singer (1911) lectured on Mind as an observable object. 

Within psychology an increasing number of experiments 
were performed without the use of introspection. Cattell 
called attention to this in 1904 in an address at the time of 
the World’s Hate inte oue “Bhomdsee 
hee, Yerkes wae deana ene gully entered by 
aa cae oping animal experimentation at 

“cagues were laying a philosophical 


A 
=. 
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justification for objectivism. Although it had not yet reached 
the experimental stage, child study was being recommended 
by Hall and Baldwin. The fields of psychopathology and ap- 
plied psychology, in which introspection is also not a satis- 
factory method, were growing in importance, and the case in 
favor of objective methods was becoming steadily stronger. 

The convergence of the three historical developments out- 
lined above gave rise in America to behaviorism. The leader 
of this movement, John B. Watson, had been a student un- 
der Angell at Chicago at the time that the concept of habit 
formation was receiving more stress. Watson, approaching 
psychology through animal study, was impressed by the 
need for objectivity. His behaviorism, launched in 1913, 
epitomized all three trends: it was an objective association 
psychology with an emphasis upon habit. It is noteworthy 
that behaviorism was fully flowered before it assimilated 
the conditioned response. In recounting his relation to Pav- 
lov, Watson writes: 


. . . I had worked the thing out in terms of HABIT forma- 
tion. It was only later, when I began to dig into the vague 
word HABIT that I saw the enormous contribution Pavlov 
had made, and how easily the conditioned response could be 
looked upon as the unit of what we had been calling HABIT. 
I certainly, from that point on, gave the master his due credit. 


The Entrance of Conditioning into Psychology 


In spite of the anticipations of conditioning in the writings 
of the Greek philosophers, in Locke, James, and others, the 
history of conditioning as such is dominated by the figures 
of Pavlov and Bekhterev. 

Ivan Petrovitch Pavlov (1849-1936) encountered the phe- 
nomenon of conditioning in the course of his studies of the 
digestive glands, for which he was awarded the Nobel Prize. 
The first report from his laboratory dealing with the newly 
discovered acquired reflexes was published in 1899 by Wolf- 
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son as a thesis entitled “Observations upon salivary secre- 
tion” (Pavlov, 1927, p. 412). These reflexes were called 
“psychic” secretions to distinguish them from the unlearned 
physiological secretions. The term conditioned reflex was first 
used in print by Tolochinov, one of Pavlov’s associates, who 
communicated his results at the Congress of Natural Sciences 
in Helsingfors in 1903. The experiments were performed in 
1901. Pavlov continued to the time of his death to carry on 
research on conditioned responses of dogs, working almost 
exclusively with salivation to food and with salivation to acid 
placed in the mouth. Because of the precision of his methods 
and his full regard for the complexity of the process, Pavlov’s 
reports, collected in two books available to English readers, 
provide a mine of information for experimenters. The facts 
are integrated by a conceptual system, which, although in 
many respects not acceptable, gives relevance to the experi- 
mental data, and poses many problems. Application of his 
findings to man was distinctly secondary for Pavlov, not be- 
cause of any lack of concern for human problems, but because 
he conceived his task to be that of brain study. Furthermore, 
he thought of his work as only in its early stages, and he rec- 
ognized its many unsolved problems. At the close of a chapter 
concerned with the applications to man he says: 

At first, not to lose si 


pelled to simplify, 
At present, after h 


ght of the main issue, we were com- 
and, so to speak, schematize the subject. 
sent aving acquired some knowledge of its gen- 
eral principles, we feel surrounded, nay crushed, by the mass 


of details, all calling for elucidation. (Pavlov, Conditioned Re- 
flexes, Oxford University Press, 1927, p. 41 1.) 

While the experimental findings of Pavlov were limited to 
the salivary responses of dogs, the concepts which he used in 
dealing with his data are those currently found in discussions 
of all types of conditioning: conditioned and unconditioned 
stimuli, reinforcement, extinction, irradiation, discrimination, 
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and so on. His work has the great merit of providing a clearly 
stated and thoroughly studied reference experiment by which 
terms may be defined and to which other experiments may be 
compared. In spite of the broad uses made of his work, Pav- 
lov continued to think of it as belonging to physiology, and 
he made no attempt to formulate a system of psychology. In 
fact, he distinguished between his work and that of the Amer- 
ican animal psychologists because he was always interested in 
the physiology of the nervous system, whereas they appeared 
to him to be concerned with essentially human problems even 
though they worked with data from animals. 

Vladimir M. Bekhterev (1857-1927) as early as 1886 had 
‘studied the localized representation in the dog’s cortex of 
learned movements such as giving the paw, but his systematic 
study of association-re flexes began in the winter of 1906-1907. 
In his experimental investigations, which used first dogs and 
later human subjects, he studied the withdrawal and respira- 
tory responses evoked by electrical shock stimulation. Unlike 
Pavlov, Bekhterev was very much interested in creating a 
new psychology. As early as 1904, before his work on asso- 
ciation-reflexes began, he delivered a lecture on “Objective 
Psychology and its Subject-Matter.” He had studied for a 
time in Wundt’s laboratory, and his writings reflect his fa- 
miliarity with the problems of psychology. It is probably to 
him more than to Pavlov that we owe the bold acceptance of 
conditioning by psychologists, although the details of condi- 
tioning which came to be accepted were Pavlov’s. Bekhterev’s 
Objective Psychology appeared serially from 1907 to 1912, and 
was translated into both French and German in 1913. The 
German translation carried the sub-title Psycho-Reflexologie, 
and Bekhterev’s position gradually became known as reflex- 
ology. In 1932 the translators of a later book were able to 
declare: “Reflexology is the dominant note in Russian psy- 
chology today, and the late Bekhterev is the guiding light of 
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humanistic science in the U.S.S.R.” The range of Bekhterev’s 
interests was enormous. He wrote widely on neurology, psy- 
chiatry, infancy, education, social psychology, judicial psy- 
chology, and aesthetics, and he founded a number of institutes 
for the study of problems in varied fields of neuropsychiatry 
and child study. 

The 1913 translations of Bekhterev’s book were very im- 
portant in arousing the interest of American psychologists, 
and most of the American experimenters preferred to use his 
motor conditioning methods rather than the salivary method 
of Pavlov. Nevertheless, both Pavlov’s terminology and his 
conceptual system have been more widely adopted than Bekh- 
terev’s. It may be that Bekhterev’s term ‘association-reflex’ 
carried with it too much of the flavor of the association of 
ideas, and thus seemed to continue a tradition against which 
behaviorism was revolting. This might be one reason why 
Pavlov’s term ‘conditioned reflex’ was more acceptable. More- 
over, Pavlov’s careful working out of one variety of experi- 
ment with great precision gave his results greater usefulness 
to laboratory workers than Bekhterev’s more general applica- 
tions. 

Early Interest in Conditioning in America. Pavlov’s 
ae Huxley lecture, delivered at Charing Cross Hospital, on 

e Scientific investigation of the psychical faculties or proc- 
esses in the higher animals” appeared in Science and was thus 
mec he nnn pe Dike anat 

oratory were summarized for presentation 


to psychologists in a thorough review by Yerkes and Mor- 
gulis in 1909. The review, howe 


mediate repetitions of Payloy’s 
published records reveal. 


ver, did not lead to any im- 
work in America, so far as 


» a student of Pavlov, performed some con- 
MENE GN Guilen porse m Gorman in 


1909 and in 1913. These came to the attention of W. H. Burn- 
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ham at Clark University, who incorporated the material into 
the lectures in his course on the mental hygiene of the school 
child. Among his students in 1913-1914 was Florence Mateer, 
who became interested in repeating Krasnogorski’s work. She 
devoted the next two years to work in this field which was 
accepted as a doctoral dissertation in 1916 and published in 
book form in 1918. Mateer’s experiments constitute the pio- 
neer study of conditioning in this country. She used the ingen- 
ious method of placing a bandage over the eyes of the child 
just before food was placed in his mouth. The bandage came 
to serve as the conditioned stimulus, eventually evoking chew- 
ing and swallowing movements before the food was presented. 
The movements were recorded kymographically from tam- 
bours fastened under the chin and on the throat. The use of 
the bandage as a conditioned stimulus was discovered more 
or less accidentally. Mateer writes: 


The great significance of the method came to me all at 
once about the fourth or fifth day of my first experiments with 
Phil, in 1914. I learned that even acceptance of a test posture, 
or entrance into the experimental laboratory, was a condition- 
ing factor and that these and other casual environmental fac- 
tors had to be unconditioned through disuse before any arbi- 
trary conditioning factor might be used as predetermined in a 
planned procedure. Even with babies who could not sit up, the 
bandage was a conditioning factor, as valuable as other stimuli 
in evoking response. Neither Bekhterev nor Krasnogorski pre- 
pared me for this, and, though I had read Pavlov, it took per- 
sonal experience to show how significant the minutiae of an 
experimental setting must be. 


Mateer’s successful conditioning of children furthered the 
case which Burnham made for the place of the conditioned 
reflex in mental hygiene (1917, 1921, 1924). 

The appearance of the translations of Bekhterev’s Objec- 
tive Psychology brought conditioning strongly to the attention 
of psychologists, and references to it began to appear in 1916. 
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H. C. Brown (1916) pointed out a possible connection be- 
tween language and the association-reflex, and F. L. Wells 
(1916) suggested that the association-reflex could supply a 
convenient account of affective transference. 

Watson’s Influence. Watson was very largely responsible 
for the enthusiastic incorporation of conditioning into Amer- 
ican psychology. Prior to his presidential address entitled 
“The place of the conditioned reflex in psychology” (1916a), 
which was delivered in 1915 before the American Psycho- 
logical Association, the conditioned reflex was not an impor- 
tant feature of American psychology. After this date, it began 
to have a prominent place in the textbooks, if not, for a time, 
in the research laboratory. 

During the winter of 1914-1915, while Mateer was work- 
ing independently at Clark University, Watson’s seminar at 
Johns Hopkins was devoted to the translation and discussion 
of the French edition of Bekhterev’s book. While Watson 
thought that Bekhterev was at heart a parallelist, rather than 
an objectivist, he did not deny the importance of his work. 
In the spring of 1915 a number of Bekhterev’s experiments 
were repeated, chiefly by Lashley who was then working with 
Watson. The experiments on finger-withdrawal, respiration, 
and heart-rate were the factual substance of Watson’s ad- 
dress. Watson used Pavlov’s expression ‘conditioned reflex’ 
thereby stamping it upon the Bekhterey type of experiment. 
The esteem in which Bekhterey was held is evidenced by the 
fact that arrangements were made for Lashley to go for a 
time to work with Bekhterev, but the World War interfered 
with this plan. 


Watson exploited conditioning in his presidential address 
primarily as a substitute for introspection. He sensed its im- 
portance, however, in relation to emotion, mental disease 
language, and habit, and wrote about these matters within 


PLACE OF CONDITIONING IN PSYCHOLOGY 13 


the next year. Its central position as the unit of habit was not 
emphasized until later. 

With one exception, Watson reported no further experi- 
ments on conditioning. In 1920 he published (with Rosalie 
Rayner) a study of the conditioning of an infant’s fear. An ex- 
periment with the boy Albert, whose fear of a loud sound was 
conditioned to a white rat and then to other furry objects, 
probably provides the most famous single case in conditioning 
literature. The way had already been paved for the intro- 
duction of conditioning into mental hygiene (Heilbronner, 
1912; Watson, 1916b; Wells, 1916; Watson and J. J. B. 
Morgan, 1917; Burnham, 1917), and this experimental illus- 
tration of a learned fear fixed a place for conditioning in later 
discussions of mental hygiene and experimental education. 

The conditioned response was prominent in Watson’s text- 
book (1919), but it had not yet displaced the chapter on in- 
stinct. By the time of his Behaviorism (1925), conditioning 
was his central theoretical concept. He had espoused a radical 
environmentalism, and had discarded instincts and other 
forms of inherited tendencies. The remaining unit of native 
behavior was the reflex; the unit of habit was the conditioned 
reflex. 

Conditioning: 1916-1926. During the first ten years after 
Watson launched the conditioned reflex, and while he and 
others were steering it to ever greater systematic importance, 
we find very few psychologists performing experiments on 
conditioning. Lashley (1916a, 1916b) gave up as unsuccess- 
ful his attempts to condition the salivary response in man. 
Mateer’s experiments had already been completed, although 
her book did not appear until 1918. Hamel (1919) concluded 
from his studies of finger withdrawal that conditioning in man 
was confused by voluntary processes, but his monograph did 
not stem the rising tide of popularity of the conditioned reflex. 
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Cason (1922a) reported that an involuntary response, not 
represented in consciousness, can be conditioned. In his ex- 
periment, the constriction of the pupil which is reflexly evoked 
by a strong light, was conditioned to a bell stimulus. He also 
conditioned eyelid reactions in human subjects (1922b) and 
his results were given a prominent place in the textbooks. Of 
the few experiments performed, several questioned the cur- 
rent broad generalizations concerning conditioning, but it 
had already been too firmly established to be dislodged by 
an appeal to the detailed data of experiments. 

The first general textbook to use a conditioning concept 
throughout was that of S. Smith and Guthrie (1921). They 
subordinated the reflex problem, and adopted the term ‘con- 
ditioned response’ instead of ‘conditioned reflex,’ broadening 
the conditioning concept to include all that had formerly been 
treated as associative learning. Allport (1924) first gave 
prominence to the conditioned reflex in a textbook of social 
psychology at a time when social psychology was seeking 
desperately for a substitute for instincts. Burnham’s Normal 
M ind (1924) presented an extended review of conditioning 
literature in a textbook on mental hygiene. In the later ’20’s, 
textbook writers in psychology, sociology, and education 
allotted increasing space and prominence to the subject. 

Conditioning: 1926-1936. In the next ten years the con- 
ditioned response lapsed into a less conspicuous rôle in text- 
books, and at the same time began to receive more attention 
in psychological laboratories. 

Ta oe importance of conditioning 
haviorism Minne hase ase T P oa omnes Dee 
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he had always thought of the conditioned reflex as one among 
many methods of experimentation, and had never, strictly 
speaking, used the method himself. With animal psychology, 
the stronghold of behaviorism, becoming more skeptical of 
this system, it was natural for the other fields in which be- 
haviorism had been strong, such as child psychology, to be- 
come more eclectic. Koffka’s Growth of the Mind (1924) 
emphasized maturation, which, as a kind of substitute for 
certain aspects of growth formerly included in instinct, now 
became a formidable competitor of conditioning as a factor 
in the development of the child’s behavior. 

As already indicated, the importance of conditioning in 
research investigation was increasing while its theoretical im- 
portance was decreasing. Several reasons for this may be 
found. The early experiments, such as those of Watson, had 
been concerned with the question: Can conditioned responses 
be formed? An affirmative answer was all that was necessary 
to sanction the theoretical use of conditioning. A few experi- 
mental examples sufficed. After the initial wave of theorizing 
experimenters began to ask: Just what are the detailed facts 
about conditioning which we can legitimately use in psychol- 
ogy? This was a question that required experimental answer. 

A second reason for the renewed interest in experimentation 
was the appearance in 1927 of a translation of Pavlov’s book. 
The Pavlov volume made available in rich detail the facts 
discovered by Pavlov and his coworkers during more than 
a quarter of a century devoted to the study of conditioned 
salivation in dogs. Pavlov’s experiments and theories called 
for repetition and confirmation using other organisms and 
other responses. Many laboratories now followed the exam- 
ple of the pioneering few which had already begun programs 
of experimentation. Liddell initiated in 1926 a series of con- 
ditioning experiments on several different animals. Kleitman 
and Crisler’s important study of conditioned salivation to 
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morphine injection appeared in 1927; Schlosberg’s study of 
the conditioned knee jerk was published in 1928; and Upton’s 
study of hearing in the guinea pig (1929) was followed by a 
series of investigations by others on the sensory capacities of 
animals. In the next few years, experimental reports on con- 
ditioned responses appeared in rapidly increasing numbers, 
and the usefulness of the method for the study of learn- 
ing, sensory acuity and related problems was clearly demon- 
strated. 

Clark L. Hull at Yale University began in 1929 a highly 
provocative series of articles in the Psychological Review 
making use of the findings of Pavlov and of other workers 
with the conditioned response in a detailed manner not pre- 
viously attempted. His presidential address before the Amer- 
ican Psychological Association in 1936 (Hull, 1937), two 
decades after Watson’s, shows the changes which have come 
in the use of the conditioning experiment. Hull, like Watson, 
stressed the conditioning process, but, in the light of the 
newer detailed knowledge of conditioned responses, he pre- 
sented a system of definitions, postulates, and theorems, by 
which to bridge the gap from conditioned responses to more 
complex forms of learning. Hull’s systematic formulations 
have led to numerous experiments by his own students and 
others. 

Because they contributed to the maturing of psychology as 
an experimental science, the developments within the years 
1926-1936 were more wholesome than those of the previous 
decade. The period was marked by a significant shift from 
the broad theoretical and speculative exploitation of condi- 
tioning to a type of theory sensitive to laboratory data. The 
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Present Conceptions of the Place of 
Conditioning in Psychology 


After abandonment of the early, extravagant claims that 
conditioning provided an immediate solution of the philo- 
sophical problems of a psychology seeking objective, scientific 
status, it became clearer that conditioning was significant 
chiefly in relation to learning theory. At the present time, 
however, there is a great diversity of opinion concerning its 
exact place in such theory. 

Largely as a result of the attacks upon associationism by 
Gestalt psychologists, contemporary learning theories tend to 
be classified into two broad groups, association theories on the 
one hand, and field theories on the other. Theories using con- 
cepts such as redintegration, trial-and-error, and condition- 
ing, are assigned to the first group; those using concepts such 
as insight, sign-learning, and least action, are assigned to the 
second group (e.g., Tolman, 1934). Field theories differ from 
conditioning and other association theories in two important 
respects. In the first place, through emphasis on the priority 
of the whole over the part, they are opposed to the analysis 
into part activities implied in conditioning theories. In the 
second place, according to field theories the dynamics of be- 
havior are said to depend on the present structure of the 
force field in which the organism is imbedded, and not on the 
previous history of the organism. Previous history may be 
useful in giving an understanding of the present, but if the 
present were completely understood, the history would be 
irrelevant. Association and conditioning theories, on the con- 
trary, have always stressed prior experience as primary for 
the understanding of present behavior. The cleavage between 
association and field theories, and attempted compromises, 
will be discussed later (Chapter 10); for the present the dis- 
tinction is important to the extent that accepting one or the 
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other positions affects the attitude taken towards the con- 
ditioned response in relation to learning. 

At least five different views may be found among those who 
accept the conditioning experiment as an example of learning. 
The conditioned response has been proposed as (1) a substi- 
tute for association, (2) the unit of habit, (3) representative 
of other forms of learning, (4) a source of deductive prin- 
ciples, (5) a subordinate and restricted form of learning. Not 
all authorities endorse specifically just one of these proposals. 
Some indeed accept two or more of them, but each of the 
proposals has been selected for separate emphasis by some- 
one (Hilgard, 1937). 

(1) Conditioning as a substitute for association. Associa- 
tion in its more objective form, in which the association occurs 
between stimuli and movements rather than between ideas, 
often goes by the name of conditioning. S. Smith and Guthrie 
gave the initial impetus to this usage in their textbook (1921). 
They extended the language of conditioning to the whole 
range of psychological phenomena. Conditioning became in 
their hands a general formula—a modern substitute for con- 
tiguity, the primary law of association. 

This usage differs from all the others in one significant re- 
spect: conditioning according to it is a single principle of 
learning, rather than a set of principles. The conditioning ex- 
periment ıs not important as a source of new principles, but 
is merely one of many instances in which the single principle 
may be applied to explain the observed modifications of be- 
havior. The other four theories agree in directing more atten- 
tion to the empirical facts discovered in conditioning experi- 
ioe ee in their conceptions of the relation of the 

ponse to learning in general. 
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maintains that the conditioned response is the unit of all 
learned activity. Thus Pavlov states: 


It is obvious that the different kinds of habits based on 
training, education and discipline of any sort are nothing but 
a long chain of conditioned reflexes. (Pavlov, Conditioned Re- 
flexes, Oxford University Press, 1927, p- 395.) 


This is the position which Watson promulgated, and it en- 
joyed a great vogue in America for some years. 

The doctrine that complex habits are simply compounds of 
conditioned responses should have led to a definite program 
of experimental work to discover the principles governing 
the chaining of component responses. Such a program did not 
follow; even the first stage of it, the development of chain 
conditioned responses, was not studied in the laboratory. 
Why Watson or his followers did not undertake the experi- 
mental investigations which followed so naturally from the 
theory requires an understanding of the place of the condi- 
tioned response in Watson’s system of behaviorism. 

The elementarism of Watson’s behaviorism was rooted in 
the scientific tradition prevailing at the time that he wrote. 
In 1916 he was combatting an introspective and sensation- 
istic analysis, but he did this within the same orthodox logical 
framework by substituting the reflex as the element in place 
of sensation, thus avoiding subjectivism, but leaving the cur- 
rent elementarism unchallenged. Sensation and attribute were 
analytic, not synthetic concepts, and the inadequacy of con- 
ditioning theory in its synthetic aspects was no greater than 
that of structuralism. It was not an important criticism at the 
time to point out that complex habits could not be completely 
reproduced by conditioning; to defend the position it sufficed 
to indicate the possibility that component conditioned re- 
sponses could be detected when the habits were analyzed. 
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A second root of Watson’s conditioning theory is found in 
his extreme emphasis upon the rôle of environment, as op- 
posed to instinct, in determining human behavior. Since habits 
cannot arise from nothing, and reflexes are the only innate 
endowment, habits must arise out of reflexes. Conditioning 
conveniently provides the principle by which reflexes become 
habits. Thus the conception of the conditioned reflex as the 
unit of habit harmonized Watson’s theoretical structure, and 
he accepted it without feeling any necessity to wait for de- 
tailed experimental verification. 

Attacks on Watson’s statement of conditioning theory have 
been numerous and vigorous. It is now clear on both the- 
oretical and experimental grounds that simple chaining of 
conditioned responses will not predict the characteristics of 
complex habits. The component responses are greatly altered 
by virtue of their combination with other responses, and the 
influence of combination must be experimentally determined. 
Watson’s simple conception of the conditioned response as 
the unit of habit has not proved serviceable and is now chiefly 
of historical interest. 

(3) Conditioning experiments as representative of other 
forms of learning. If the conditioning experiment proves to 
exhibit in sharp focus all the essential characteristics of other 
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underlying process. Symonds’ analogies show many unjusti- 
fiable leaps from conditioning to more familiar behavior. 

All learning cannot be epitomized in conditioning, because 
the conditioning experiment does not have within it all the 
richness of patterning of stimuli and of coordination of move- 
ment found in other learning situations. To point out an anal- 
ogy with conditioned finger retraction can never suffice to 
account for the most favorable method of learning even a 
manual skill such as typing, much less for more varied in- 
stances of problem-solving. This use of the conditioned re- 
sponse as representative of other forms of learning is properly 
limited to situations in which the identity is very apparent or 
is susceptible to demonstration. 

(4) Conditioning as a source of deductive principles. The 
relationships within conditioning experiments do not in them- 
selves provide analogies to account for the backward order 
of elimination of blind alleys in rat mazes or for the excess of 
errors in the middle of a memorized series of nonsense sylla- 
bles. Yet it may be possible to deduce facts such as these from 
principles discovered in conditioning experiments, if the pro- 
cedure advocated most prominently by Hull is followed. This 
position differs from the previous ones in that the deduced 
behavior may be superficially very unlike the conditioned re- 
sponse behavior which serves as the source of the principles 
used in the deduction. In the hands of Hull and others, the 
conditioning experiment has become the tool for harmonizing 
many facts already known about learning, and for predicting 
a variety of new facts. 

This use of conditioning does not carry the implication that 
all habits are compounded of simple conditioned responses or 
that the principles discovered in conditioning experiments are 
directly applicable to complex habit situations. The condi- 
tioning experiment, because of its simple and well controlled 
structure, is a fruitful source of postulates from which deduc- 
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tions can be made concerning complex learning. The postu- 
lates must be verified in their new application; it is not enough 
to know that they were found true in simple conditioning. 

(5) Conditioning as a subordinate and restricted form of 
learning. Psychologists who believe that each of the preceding 
points of view is either wrong or premature fall into two 
classes: first, those who are opposed to all association theories 
of learning, and second, those who are not opposed to asso- 
ciation theories, but who believe that the conditioned response 
has been overemphasized. In the former group would fall the 
Gestalt psychologists, who have been most vigorous in their 
denunciation of associationism. They very naturally include 
the conditioned response among the targets of their attack, 
because it seems to possess the vices both of association the- 
ories of learning and of atomistic-mechanistic theories of 
science. Even they need not deny some minor place to com 
ditioning as a laboratory curiosity, but they cannot be ex- 
pected to endorse a conditioning theory of learning. Others, 
such as Robinson (1932), who defend an associationist ac- 
count of learning believe that the conditioned response experi- 
ment is too limited to provide all of the principles needed to 
deal with more complicated learning. They would also be 
skeptical about a theory of learning based solely on the maze 
experiment or solely on serial memorizing. There is good 
reason to expect a richness of empirical learning principles 
commensurate with the variety of situations in which learn- 
ing occurs. 

No attempt has been made here to evaluate the theories 
summarized above, or to do more than place them in the 
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cise. The first wave of enthusiasm has passed; assuredly the 
conditioned response is taking a more modest place than it 


filled ten years ago. 


Notes 


Preparation within Psychology 
for the Acceptance 
of Conditioning 


Early illustrations of concepts an- 
ticipating the conditioned response 
may be found in reviews by Cason 
(1925a) and Hull (1934c). 

The development and influence of 
association psychology is compre- 
hensively traced in H. C. Warren’s 
A History of Association Psychology 
(1921). The rise of objectivism is 
treated in the standard histories of 
psychology, such as Boring’s His- 
tory of Experimental Psychology 
(1929) and Murphy’s An Historical 
Introduction to Modern Psychology 
(1929). 

Acknowledgment is made to Dr. 
Watson for permission to quote from 
a personal communication regarding 
his relationship to Pavlov. It is in- 
teresting to note that in Russia, as 
well as in America, objectivism 
gained a strong position prior to the 
advent of conditioning. Russian ob- 
jectivism is largely derived from 
Sechenov, a distinguished physiolo- 
gist, who published his Reflexes of 
the Brain in 1863. In this book he 
outlined a comprehensive reflex the- 
ory of all behavior, including cere- 
bral processes. Sechenov’s concep- 
tions were never well known outside 
Russia, but both Pavlov and Bekh- 
terev acknowledge his great influ- 


ence on their work. Recently a 
translation of the most important 
works of Sechenov has been made 
available in a volume (1935) issued 
at the time of the International 
Congress of Physiology in Moscow. 


The Entrance of Conditioning 
into Psychology 


Pavlov’s work is summarized in 
two books available in English 
translation (1927, 1928). A biogra- 
phy is included in the second of 
these. Froloy’s more recent book 
(1937) is useful chiefly for its char- 
acterizations of Pavlov as a man. 

Bekhterev’s work is summarized 
in his Objective Psychology, which 
appeared simultaneously in French 
and German (1913a, 1913b), and 
in his General Principles of Reflex- 
ology, which appeared in English in 
1932. This contains a biographical 
statement, and a bibliography of his 
writings. 

It is difficult to estimate the ex- 
tent to which the lagging interest in 
experimental work on conditioning 
in America may be attributed to in- 
accessibility of the Russian publica- 
tions. Yerkes, having been early in- 
terested in Pavlov’s work, and hav- 
ing himself done independently some 
experiments on the frog (1905, 
1906) which Pavlov considered pi- 
oneering work along the lines of 
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conditioning, had urged Pavlov as 
early as 1909 to prepare a book for 
the Animal Behavior Series which 
Yerkes edited. Pavlov’s reply (pub- 
lished here with Dr. Yerkes’ permis- 
sion) is characteristic: 

“For a long time I have written 
unwillingly and rarely, and the pro- 
posed writing for you would not be 
easy for me. Frankly, I am planning 
to write a book on the conditioned 
reflexes, but only later when the 
subject shall have been carried far- 
ther and entirely worked over. The 
physiology of the conditioned re- 
flexes is for me only a half of the 
story. Already, we work over the 
cord reflexes in connection with most 
varied destructions of the hemi- 
spheres. The objective of the whole 
work, as I understand it, is a foun- 
dation for the physiology of the 
brain.” 

Dr. Mateer kindly consented to 
the publication of a private com- 
munication regarding her early work 
under Burnham. In the course of 
preparation for her dissertation, she 
borrowed the Russian edition of 
Sechenov’s Reflexes of the Brain 
from the Surgeon General’s Office, 
and had significant passages trans- 
lated. The interest at Clark Univer- 
sity was serious, and quite inde- 
pendent of that at Johns Hopkins. 
Baird, who supervised the experi- 
mental portions of Mateer’s study, 
arranged for her to meet Watson in 
1914 because of their common in- 
terest in conditioning, but it is her 
recollection that at the time Watson 
did not mention experiments in 
Progress. Lashley (private commu- 
nication) agrees in the priority of 
the Clark experiments. 


The account of the introduction 
of the conditioned reflex at Johns 
Hopkins is based upon communica- 
tions from Watson and Lashley. 
Watson credits Lashley with most 
of the early experimentation; Lash- 
ley states that the initiative was 
Watson’s in all that was done. Dr. 
Lashley writes, in part: 

“We accumulated a considerable 
amount of experimental material on 
the conditioned reflex which has 
never been published. Watson saw 
it as the basis for a systematic psy- 
chology and was not greatly con- 
cerned with the mechanism of the 
reflex, which he regarded as a prob- 
lem for the physiologist. I got in- 
terested in the possibility of tracing 
conditioned reflex paths through the 
cortex and this started my program 
of cerebral work.” 


Present Conceptions of the 
Place of Conditioning 
in Psychology 


The use of conditioning as a sub- 
stitute for the law of contiguity has 
received further treatment by Wil- 
son (1924) and by Guthrie (1930, 
1935, 1938). A somewhat similar 
proposal was made by Frank (1923) 
and promptly criticized by Peterson 
(1923). Analogous statements of an 
objective associationism have been 
proposed, such as Meyer’s principle 
of double stimulation (1908, 1934), 
Washburn’s principle of the associa- 
tion of movements (1916), and Hol- 
lingworth’s principle of redintegra- 
tion (1928). Pavlov (1932b) has 
castigated the use of a single princi- 
ple such as this to account for all 
learning, because it results in over- 
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looking many important concrete 
facts of conditioning. For a reply to 
Pavlov, see Guthrie (1934). 

The statement that the condi- 
tioned reflex is the unit into which 
habits may be resolved was made in 
1917 by Watson and J. J. B. Mor- 
gan. It was frequently reiterated by 
Watson (1919, 1926), and quoted 
with approval by others, e.g., Burn- 
ham (1924). Allport (1924) intro- 
duced essentially the same notion 
into social psychology. The infiltra- 
tion of this point of view into the 
literature of education, sociology, 
and mental hygiene may be traced 
largely to these sources. 

Symonds’ exposition of condition- 
ing as representative of all forms 
of learning became widely known 
through Sandiford’s (1928) text- 
book on educational psychology. A 
number of other authors have fa- 
vored a similar conception, e.g., 
Hunter (1933), Rexroad (1932, 
1933), and Stagner (1931). 

The laws of behavior recently 
proposed by Skinner (1938) are 
based on the detailed experimental 
study of a particular type of repre- 
sentative behavior, to be discussed 
later (Chapter 3). Skinner has made 
so few extensions of his principles to 
other situations, however, that it is 
not quite clear to which of the five 


positions his point of view should be 
assigned. 

Hull’s numerous papers in which 
he details the deductive approach 
from conditioning to learning theory 
may be located from the list of ref- 
erences. The most useful single pres- 
entation of his program may be 
found in his presidential address 
before the American Psychological 
Association (1937), and the most 
extensive application to the phe- 
nomena of rote memorizing is con- 
tained in a recent monograph by 
Hull, Hovland, Ross, M. Hall, D. T. 
Perkins, and F. B. Fitch (1940). 

Illustrations of the criticisms of 
the association position in learning 
may be found in Kohler (1929), 
Peterson (1935), Tolman (1932), 
Wheeler (1929). Many accept the 
conditioning experiment as an in- 
teresting but limited situation, dis- 
tinctly not representative of all 
learning. Among these are writers 
representing widely diverse theoreti- 
cal points of view, e.g, Adams 
(1931), Dodge (1933), Finch and 
Culler (1935), Harlow (1936), 
Humphrey (1928a, 1933), Lashley 
(1916a, 1934), Stephens (1931b), 
Thorndike (1932a, 1935), Warner 
(1932b), K. A. Williams (1929b), 
Gustav and Wolf (1937). 


Chapter 2 
CLASSICAL CONDITIONING EXPERIMENTS 


EFORE a consideration of the principles of conditioning is 
B undertaken, the results of the experiments upon which 
those principles are based must be surveyed. The basic rela- 
tionships of any learning theory are derived from experiments 
which have been designed to display in relatively simple and 
uncomplicated fashion the fundamental processes which are 
involved. Since learning is essentially a change in an individ- 
ual’s behavior which depends upon his previous experience 
in the same situation, it is apparent that in any learning ex- 
periment the individual must be observed at least twice. The 
progress of learning is usually followed by recording the be- 
havior during successive trials, or discrete presentations of the 
situation. The most serviceable experiments for the demon- 
stration of the fundamental principles of learning will be 
those in which all the factors influencing behavior are well 
controlled or measured, and in which simplicity is secured by 
studying a single response rather than a complicated series of 
responses. The conditioning experiment fulfills these require- 
ments. In this respect it may be considered a basic experiment. 
There are, of course, a number of more complicated learning 
situations not readily reducible to the pattern of conditioning. 
Among them are serial performances such as maze-learning 
ee r eee involving tools and de- 

g and reasoning experiments, and 


skilled acts involving precision and patterning of voluntary 
26 
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movements. The relation of conditioning to these further 
situations will be considered later (Chapters 8-10), after the 
problems involved in the simpler situations have been more 
fully analyzed. 

The body of data accumulated through years of study of 
conditioned salivation in the dog by Pavlov and his coworkers 
serves today as the point of departure for a description of 
conditioning experiments. Pavlov’s work defines the type of 
situation to be considered as conditioning, and provides a 
terminology for its description. It is obvious that condition- 
ing is not, however, restricted to the salivary responses of 
dogs, for it is entirely feasible to use other organisms and 
other responses without departing widely from the Pavlovian 
tradition. The essential features of the conditioning experi- 
ment are the following: (1) an unconditioned stimulus which, 
in the experimental situation, evokes a regular and measur- 
able unconditioned response, (2) a conditioned stimulus which 
originally does not evoke the unconditioned response, (3) 
repeated presentation of the conditioned and unconditioned 
stimuli in a controlled and specified manner. Any new or 
altered response to the conditioned stimulus whose occur- 
rence depends upon the repeated double stimulation is called 
a conditioned response. 

Within this general structure, however, certain important 
differences in procedure may be found among various condi- 
tioning experiments. Two general types of experiment will be 
discussed: (1) classical conditioning, and (2) instrumental 
conditioning. The first type is the experiment of Pavlov, but 
the other type has also been studied by his students and has 
come to be designated as conditioning. The distinction be- 
tween the two training procedures is based on the conse- 
quences of the conditioned response. In the first type the 
occurrence of the conditioned response results in no change 
in the procedure; the unconditioned stimulus is presented in- 
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variably at a fixed interval after the conditioned stimulus. In 
the instrumental training, however, the conditioned stimulus 
is followed by a reward (or an avoidance of punishment) only 
when the appropriate response is made to it. Many learning 
situations represent a combination of these types. Discussion 
of situations other than classical conditioning is postponed to 
the next chapter. 

The reference experiment for classical conditioning is the 
study of conditioned salivation in dogs. The essential features 
of the procedure may be seen from a description of a typical 
experiment in Pavlov’s laboratory (Anrep, 1920). In order to 
make possible the recording of the magnitude of the salivary 
response, the dog was first subjected to a minor operation in 
which the duct of the parotid gland was diverted so that the 
saliva flowed through an opening on the outside of the cheek. 
A small glass funnel was firmly cemented over the opening to 
collect the saliva, which could be measured with an accuracy 
of one-tenth drop by suitable devices. The dog was trained to 
stand quietly in a loose harness on a table in a room which 
was insulated against any distracting noises or vibrations. The 
experimenter occupied an adjoining room, observing the dog 
through a small window, and presenting the stimuli by means 
of automatic devices. A tuning fork was sounded and 7 or 8 
seconds after the beginning of this conditioned stimulus a 
plate containing a small measured quantity of dry powdered 
food was moved within reach of the dog’s mouth. No saliva- 
tioniwas evoked by the tone, but during the eating there was 
wan fof salva, Combinations ofthe tone and fod 
intervals of from 5 to 35 mi ae ae Ree ae Be 
tioned response was hearer ; ees eeo 
for 30 seconds and measurin a re rae ale ue 
Io double stimulations eee À Eo a a ee ee 
eae ae as a slight conditioned saliva- 

? 3° combinations the tone evoked a salivation 
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of 60 drops. On the early tests the conditioned salivation did 
not begin until the tone had sounded for 18 seconds; on later 
tests the salivation commenced after only I or 2 seconds. 

Another example of classical conditioning which employs 
a motor rather than a secretory response may be selected from 
Liddell’s (1934) studies on sheep. A metronome was set beat- 
ing once per second for five seconds and then a shock was de- 
livered from an inductorium through electrodes attached to 
the left foreleg of the animal. The metronome had no appar- 
ent effect upon the sheep’s behavior, whereas the shock evoked 
a definite brief flexion of the limb, accompanied by changes in 
respiration and in electrical resistance of the skin. After 4 
presentations of the metronome and shock in combination, 
the conditioned stimulus caused a definite change in breath- 
ing and skin resistance, and on the 6th trial there was a slight 
movement of the leg. By the 11th trial the conditioned leg 
flexion was fully established, and with further training was 
quite constant and stable. The reactions of the 11th trial are 
illustrated in Figure 1. Since the conditioned response oc- 
curred at a latency of 3 or 4 seconds following the onset of the 
metronome, it was not necessary to change the training pro- 
cedure in any way to measure the strength of response on 
each trial. 

Conditioning has been successfully carried out on many 
different organisms, using a large number of different un- 
conditioned responses and conditioned stimuli. Its apparent 
universality supports the use of conditioning by many psy- 
chologists as a sort of common denominator of all learning 
situations. An enumeration of the various organisms and re- 
sponses which have been studied will suggest both the gen- 
erality and the limitations of conditioning phenomena. 

Organisms Which Have Been Conditioned. Simple as- 
sociations, which may be called conditioned, have been formed 
by protozoa, worms, snails, crabs, fish, reptiles, pigeons, 
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chickens, rats, sheep, dogs, monkeys, and many other species. 
A few illustrations will serve to indicate the kinds of experi- 
ments done with animals differing widely in anatomical and 
behavioral characteristics. 


Une R 
| | 
Left foreleg 


Respiration 


Metronome C a a = ere 
Shock (Unc s) ee i OE 


FIGURE 1 
CONDITIONED RESPONSES IN THE SHEEP 


The diagram represents a tracing from a kymograph record of the eleventh 
presentation of a metronome (conditioned stimulus, CS) combined with a 
shock (unconditioned stimulus, UncS) delivered to the left foreleg. The 
metronome beat once a second for § seconds prior to the shock. Both the 
conditioned leg movements and respiratory changes began while the metro- 


nome was sounding, anticipating the shock. Th diti 
(GR) may be distinen hed Gam th Finca pe nati Oo 
lowing shock. After Liddell (1934, ae ae eapopies UES ae 


Experiments on colonies of infusoria, Carchesi lachmani 
were performed by Plavilstchikov (1928) and siramariced 
statistically by Razran (1933a). Tactile stimulation with a 
glass filament induced contraction of the colony (the uncon- 
ditioned response). A filtered light, to which no original re- 
sponses were observed, served as the conditioned stimulus. 
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Reactions were observed through a binocular microscope. 
Conditioned responses were established in all 82 colonies ob- 
served. The first response appeared after 79 to 284 repe- 
titions. Parts of the conditioned colony were in several cases 
transplanted to hosts not previously used in the experiment. 
The new colony developed contractions after I to 3 trials. 
When the transplanted parts were removed after a few more 
trials, the remaining parts of the host continued to show con- 
ditioned responses after 1 to 3 additional trials. 

Movements of the mouth region of the snail, Physa Gyrina 
Say, were conditioned by E. L. Thompson (1917). After the 
snails were adapted to the circumstances of experimentation 
a piece of lettuce pressed against the mouth region resulted 
frequently in mouth movements. Pressure applied to the ven- 
tral surface of the foot, not originally eliciting mouth move- 
ments, served as the conditioned stimulus. After frequent 
combinations of the pressure with the lettuce, mouth move- 
ments to pressure were elicited as conditioned responses. Ex- 
perimental extinction was demonstrated when the pressure 
was applied a number of times without being followed by the 
lettuce leaf. 

A number of experimenters have established conditioned 
responses in fish. Frolov (1925a), a student of Pavlov’s, de- 
signed a method for transmitting the unconditioned stimulus 
(electric shock) to the fish by a wire clamped to the dorsal 
fin. This wire, attached to a Marey tambour, also recorded 
the movements of the fish on a kymograph. Conditioned 
stimuli were presented by the use of lamps, a submerged 
telephone, and an electric bell and tuning fork suspended 
above the aquarium. Conditioned motor responses appeared 
after 5 to 30 combinations of conditioned and unconditioned 
stimuli, best results being secured with the submerged tele- 
phone. 

When similar sensory and muscular processes are chosen 
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in different species, it is possible to perform experiments which 
are useful for comparative purposes. Thus eyelid reactions 
have been conditioned in rats (Hughes and Schlosberg, 1938), 
dogs (Hilgard and D. G. Marquis, 1935), monkeys (Hilgard 
and D. G. Marquis, 1936), and men (Hilgard and A. A. 
Campbell, 1936). In each of these studies, the unconditioned 
response was lid closure to a puff of air directed at the eye. 
When the conditioned stimulus (a buzz in the case of the 
rat, a light in each of the other studies) preceded the air-puff 
regularly by a fraction of a second, the conditioned response 
developed in each case as a lid movement following the on- 
set of the conditioned stimulus but anticipating the air-puff. 
Records from dog, monkey, and man are reproduced in Fig- 
ure 2. The gross similarity of the results in different species 
must not be permitted to obscure the differences. Although 
the air-puff is delivered to one eye only, the response tends 
to be equal in both eyelids in the monkey, more nearly con- 
fined to the puffed eye in the dog, and intermediate in man. 
In the human record illustrated, the conditioned response is 
similar in both eyelids, although the unconditioned response 
has habituated somewhat in the non-puffed eye and is of less 
amplitude than that of the puffed eye. 

Reactions Used as Unconditioned Responses. In order 
to carry out a conditioning experiment it is first necessary to 
discover in the repertory of the organism some dependable 
unconditioned response on which to base the conditioning. 
Table 1 lists the responses which have been most used, and 
the unconditioned stimuli employed to elicit them. 

It is obvious that the classificatory scheme is not a rigid 
one, for responses of an intact organism are too complicated 
to be assigned with assurance to reflex or 
gories. The responses in the first group are u 
of the autonomic nervous system. These ma 
show certain differences, such as greater lat 
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pared with responses of the second group, which are under the 
control of the somatic nervous system. In man, the awareness 
of the responses may also differ. The human subject is not 
ordinarily as conscious of his pupillary changes or salivary 
secretions as he is of his overt muscular movements. On the 
other hand, the autonomic responses are affected by states of 
apprehension and emotion of which the subject is clearly 
aware. There is not, therefore, a sharp dividing line between 
smooth muscle and striped muscle responses so far as psy- 
chological determination of response is concerned. 


TABLE I 


UNCONDITIONED RESPONSES AND THE STIMULI Usep tro Exicrr THEM 


1. Glandular, smooth muscle, and blood responses: 


Salivation Dry food; acid 
Change in skin resistance Electric shock 
Pupillary reflex Change in illumination 
Gastro-intestinal secretions Food 
Vasomotor reactions Shock; thermal stimuli 
Nausea, vomiting, etc. Morphine 
Immunity reactions Injection of toxin, antigen 
Diuresis Increased water intake 

2. Relatively involuntary responses in striate muscle: 
Flexion reflex Electric shock 
Knee jerk Patellar blow 
Eyelid reflex Shock; sound; air-puff 
Eye movements Rotation 
Change in respiration Electric shock 
Change in pitch of voice Electric shock 

3. Semt-voluntary and voluntary responses: 
Withdrawal movements Electric shock 
Mouth opening, swallowing Food 
Locomotion Shock 
Instructed responses Various 


Previously conditioned responses Various 


The difference between responses of the second and third 
groups is likewise one of degree. To state that a response is 
voluntary means that its elicitation is highly dependent upon 
the instructions given to the subject, whether by command or 
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by previous training. Thus the instruction to press a telegraph 
key at a signal yields a voluntary response, although the re- 
sponse may become relatively automatic with practice. A less 
voluntary response such as the knee jerk depends more upon 
the eliciting stimulus than on prior instructions. Even the 
most involuntary responses are subject to facilitation or inter- 
ference by voluntary sets and attitudes of the organism, and, 
conversely, the most voluntary responses are influenced by 
the characteristics of the stimuli which set them off. The dif- 
ference is primarily one of degree. In most conditioning ex- 
periments a relatively involuntary reaction is selected as the 
unconditioned response because it is more directly under ex- 
perimental control. When an instructed reaction is employed 
additional factors, difficult to measure precisely, must be taken 
into consideration. The basic principles of conditioning can 
best be discovered in the simpler situation, and the data of 
voluntary conditioning are postponed for later discussion 
(Chapter 11). . 
The use of previously conditioned responses as uncondi- 
tioned responses is designated higher-order conditioning. An 
experiment from Pavlov’s laboratory illustrates the sequence 
of events. A metronome was combined with food until condi- 
tioned salivation to the metronome was established. Then a 
black square was held before the dog for 10 seconds, and 
after an interval of 15 seconds the metronome was sounded 
for 30 seconds. At the tenth repetition the dog began to 
salivate to the black square. The metronome had served as 
the unconditioned stimulus. Special precautions are needed 
in establishing such second order conditioned responses be- 
cause the response to the metronome tends to be extinguished 
if it is not followed by the unconditioned food stimulus, and 


because a new stimulus (black square) presented along with 
the metronome under these 


le ) conditions tends to become a con- 
ditioned inhibitor. The most effective procedure, according to 
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Pavlov (1927, p. 33), is to withdraw the new stimulus some 
seconds before the primary stimulus (metronome) is pre- 
sented. 

Extreme difficulty or failure has been encountered in the 
attempt to condition certain responses. These failures appear 
to fall into three classes. (1) Conditioning has been shown to 
be unsuccessful in vertebrates if the unconditioned response is 
evoked otherwise than by way of the central nervous system. 
Passive flexion of a limb, or direct electrical or chemical stim- 
ulation of a muscle or its motor nerve are ineffective as un- 
conditioned responses (see p. 322, Chapter 13). (2) Certain 
simple reflexes appear to be extremely difficult to condition. 
The abdominal, patellar, Achilles, and plantar reflexes fall in 
this category. When successful conditioning does occur, as in 
the case of the knee jerk (Schlosberg, 1932), it appears that 
voluntary habitual facilitation is an important factor. 6) 
Some conditioning experiments are unsuccessful because of 
the presence in the situation of strong antagonistic tendencies 
which interfere with the performance of the conditioned re- 
sponses. Several experimenters have reported failure to secure 
conditioned finger retraction based on unconditioned shock 
stimulation because of a competing tendency which is estab- 
lished by general instructions to hold the fingers on the elec- 
trodes. Further illustrations of this phenomenon will be pre- 
sented in other connections. 

Conditioned Stimuli. While unconditioned stimuli are 
chosen in reference to the response to be elicited, any environ- 
mental change to which the organism is sensitive may serve 
as a conditioned stimulus. Visual stimuli have included lights 
of various colors, papers, geometrical forms, rotating objects. 
Auditory stimuli have included pure tones, horns, buzzers, 
bubbling water, metronomes. Various thermal, tactual, olfac- 
tory, and proprioceptive stimuli have been used. Stimulus 
patterns may be presented in temporal as well as in spatial 


36 CONDITIONING AND LEARNING 


arrangements. Pavlov tended to prefer continuing stimuli, 
such as metronomes, electric fans, rotating disks. Experi- 
menters using reflexes in striate muscles have tended to prefer 
stimuli with sharp onset, such as flashes of light, clicks, con- 
tact stimuli. The effectiveness of some stimuli may depend 
on their sudden onset, while others may depend for their 
effectiveness on summation. Pavlov has shown that a change 
in the intensity of a stimulus or the termination of a stimulus 
may serve as well as the appearance of a stimulus. 

Internal stimuli are not as frequently employed in condi- 
tioning experiments as external stimuli because they are not 
so susceptible to precise control and measurement. They are 
important, however, in many types of experiment, as the 
technique of temporal conditioning illustrates. If a dog is 
given food regularly every thirtieth minute, it comes to sali- 
vate regularly at that interval. If the food is omitted the dog 
will salivate approximately 30 minutes after the last feeding. 
Although their nature has not been determined, it is apparent 


that internal stimuli of some sort are functioning as signals 
for the response. 


The Nature of the Conditioned Response 


The conditioned response has traditionally been assumed 
to be the response originally elicited by the unconditioned 
stimulus, evoked after learning by the conditioned stimulus. 
This assumption requires that the dog which formerly sali- 
vated only to the food should salivate in the same way to the 
metronome; the human subject who gave a galvanic reaction 
to an electric shock should respond similarly to a flash of light 


fee conditioning. It is now recognized, however, that con- 
tioned and unconditioned responses are seldom 
? 


pais and that the conditioned response is not s 
uplicate of the unconditioned response. Some of th 
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ences are illustrated in Figure 3. Many experiments offer little 
critical evidence on the nature of the conditioned responses, 
because the only responses measured are those similar to the 
unconditioned response. Thus Pavlov measured only salivary 
secretion, and neglected any precise measurement of the mo- 
tor concomitants. When attention is directed to the differences 
between conditioned and unconditioned behavior, as well as 
to similarities, the conditioned response is found to be very 
complex, and to display significant variations from one experi- 
ment to another. Four different kinds of conditioned response 
will be described. 

Redintegrative Responses. In some experiments, the con- 
ditioned response does appear to be almost an exact replica 
of the unconditioned response. Thus in the early stages of 
conditioned leg withdrawal, dogs act in the presence of the 
conditioned stimulus as though they were being shocked 
(Culler, Finch, Girden, and Brogden, 1935; Kellogg, 1938a). 
The mouth-opening, sucking, and swallowing of infants con- 
ditioned in the first 10 days of life (D. P. Marquis, 1931) 
appear to duplicate the unconditioned responses to the nurs- 
ing bottle. One of the more dramatic illustrations is provided 
by the conditioned reactions based on morphine injection. 
The conditioned responses, in the absence of morphine, may 
include profuse salivation, vomiting, and other reactions char- 
acteristic of the nausea produced by morphine itself (Kleit- 
man and Crisler, 1927). 

Fractional Component Responses. In many experiments 
it is found that the conditioned response is merely a compo- 
nent of the unconditioned response. Thus conditioned saliva- 
tion may appear without chewing, or conditioned chewing 
movements without swallowing, although the complex un- 
conditioned response included salivation, chewing, and swal- 
lowing. A dog which has received a shock on the leg together 
with a buzzer may give a conditioned withdrawal response 





Ficure 3 

Decree or RESEMBLANCE oF ConpiTio 
The four records 
responses of a single human 
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photographic records of the eyelid 
subject. The top record shows the reflex to 


are tracings from 
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without the vocalization which was part of the reaction to 
shock. The most characteristic finding of conditioning experi- 
ments is that the conditioned response resembles, but does not 
completely duplicate, the unconditioned response. 
Preparatory Responses. A number of writers have called 
attention to the function of the conditioned response as a 
preparation for the unconditioned stimulus. In certain cases 
the conditioned response does not resemble the unconditioned 
response at all. It seems that the subject is getting ready for 
the forthcoming stimulation, and acting as if he were expect- 
ing it. The preparatory character of conditioned behavior in 
the Pavlov situation has been described by Zener (1937). 
Salivary responses were conditioned to a bell which preceded 
feeding in the usual manner. The total behavior was recorded 
by motion pictures, and the differences between conditioned 
and unconditioned behavior were summarized as follows: 


Except for the component of salivary secretion the condi- 
tioned and unconditioned behavior is not identical. (a) Dur- 
ing most of the time in which the bell is reinforced by the 
presence of food, chewing generally occurs with the head 
raised out of the food-pan but not directed either at the bell 
or into the food-pan, or at any definite environmental object. 
Yet this posture practically never, even chewing only occa- 
sionally, occurs to the conditioned stimulus alone. Despite 
Pavlov’s assertions, the dog does not appear to be eating an 
imaginary food. (b) Nor is the behavior that does appear an 
arrested or partial unconditioned reaction consisting of those 
response elements not conflicting with other actions. It is a 





light (Ru) and the unconditioned response to the air-puff (UncR) before 
conditioning. In the second record the slight anticipatory conditioned 
response (CR) might be considered a reduced replica of the unconditioned 
response. In the third record, the irregularity and recruitment of the con- 
ditioned response differ markedly from the characteristics of the uncondi- 
tioned response. In the final record, the complete lid closure prevents the 
air-puff from reaching the eye. At this stage, the response may be classified 
as an avoidance reaction, as described more fully in Chapter 3. The original 
records were reproduced by Hilgard (1936a). 
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different reaction, anthropomorphically describable as a look- 
ing for, expecting, the fall of food with a readiness to perform 
the eating behavior which will occur when the food falls. The 
effector pattern is not identical with the unconditioned. (c) 
Movements frequently occur which do not appear as part of 
the unconditioned response to food: all the restless behavior 
of stamping, yawning, panting. (Zener, 1937, p. 393-) 


When the restraining straps were removed, so that the 
range of the dog’s activity was increased, the preparatory 
character of the behavior became more evident. At the con- 
ditioned signal for food, the dog would approach the food 
pan; at another signal which had been associated with the 
release of acid into the mouth, the dog would either do noth- 
ing (since the acid delivering tube was not attached), or it 
would walk away from the neighborhood of the stimulating 
devices. Furthermore, when the dog was satiated with food, 
the reactions to the conditioned stimulus were not only de- 
creased in amount but qualitatively altered. 

Even when the conditioned response appears in the same 
effectors as those of the unconditioned response, the form of 
response is often sufficiently different to make an interpreta- 
tion as a fractional component response inappropriate. Thus 
conditioned respiration appeared in guinea pigs as restrained 
breathing, while the unconditioned response to the shock was 
a sharp inspiration (Upton, 1929). 

To call a conditioned response a Preparatory or expectation 
response 1s not to explain it. It means merely that the experi- 
menter considers the response to be an appropriate reaction 
in the total situation. The origin of the appropriate behavior 
remains undescribed and unexplained. It must be assumed 
ee ae n a from its potential behavior 

able to the occasion. While 


something of the sort happens in conditioning experiments, 
many difficult problems of explanation are raised. 
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Modifications not Depending upon Double Stimula- 
tion. Within a conditioning experiment there are some pro- 
gressive changes in response not qualifying strictly as condi- 
tioning, yet easily confused with it. In young organisms, for 
example, changes during a few days of experimenting may be 
due to the maturing of sensory and response mechanisms, and 
not to the circumstances of experimentation. In other cases, re- 
petitive stimulation produces decrements commonly described 
in terms of adaptation or fatigue. To the extent that these 
changes would have occurred through the repeated presenta- 
tion of single stimuli (not depending, therefore, upon the 
relationship between conditioned and unconditioned stimuli) 
they are not to be considered as changes truly representative 
of conditioning. Since such changes occur within conditioning 
experiments, control observations are necessary to determine 
the extent to which the changes are dependent upon the par- 
ticular arrangements of the conditioning experiment. 

There are certain phenomena, even more closely allied to 
those of conditioning, which occur in experiments in which the 
pairing of conditioned and unconditioned stimuli has been 
deliberately avoided, so that the requirements of true con- 
ditioning are not met. After a reflex to light had been adapted 
in fish, so that the light no longer evoked a response, the fish 
were shocked repeatedly, no illumination change accompany- 
ing the shock. After this regimen, both light and a vibratory 
stimulus produced reactions, which might have been inter- 
preted as conditioned if the light and vibration had been com- 
bined with the shock (R. R. Sears, 1934). Similarly, monkeys 
were frightened several times by a powder flash (or a “snake” 
blow-out). Following this series of experiences, a bell was 
sounded, and fright responses were evoked which the bell 
had not previously elicited (Grether, 1938). The process has 
been described as pseudo-conditioning, since results similar 
to conditioning are obtained without following a strict condi- 
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tioning procedure. The pseudo-conditioning consists in a 
heightened state of excitement, so that the animal is sensitized 
to stimuli not normally arousing the response. 

The augmentation of an original response to the conditioned 
stimulus through a conditioning procedure may be described 
as sensitization. This is merely a form of pseudo-conditioning, 
unless it can be demonstrated that the sensitization is a func- 
tion of the repetition of conditioned and unconditioned stimuli 
in precise relationship. Wendt (1930) was able to detect by 
a facilitation technique that there was a subthreshold tend- 
ency to give a bilateral knee jerk at a latency of 120 to 180 
msec. after a tap on one leg. This is longer than the latency 
of the usual knee jerk (30 to 50 msec.). Subsequently a tap on 
the tendon of one leg was followed after 200 msec. by a tap 
on the other, and the bilateral reflex detected in the facilita- 
tion experiment appeared as a conditioned response, and in- 
creased in magnitude as training was continued. This response 
alteration, termed sensitization, depends for its nature on an 
original subthreshold tendency to react in that manner to the 
conditioned stimulus. Another illustration is provided by the 
startle response as conditioned in the rat by Prosser and 
Hunter (1936). Auditory stimuli just too weak to elicit startle 
responses were paired with an electric shock. This procedure 
increased the excitability to a point at which the reflex re- 
sponse was elicited by the previously ineffective sound. The 
authors doubted that true conditioning had occurred, since 
the latency of the ‘conditioned’ response was the same as that 
of the unconditioned response to more intense sounds. In a 
later experiment, Hunter (1937) showed that true condi- 
tioned startle reactions differed in latency and in other char- 


acteristics from the unconditioned startle response. Sensitiza- 
ton is encountered only very rarely and is 
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evidence that the phenomenon depends upon the temporal 
relation of the stimuli as the usual conditioned response does 
(Bernstein, 1934). Wendt found, for example, that instruc- 
tions to kick voluntarily to the tendon tap also sensitized the 
bilateral reflex. A state of apprehension or excitement is 
common enough in conditioning situations, and if such a state 
produces exaggerated reflexes they may be thought of as 
accessory to conditioning, rather than as illustrations of ordi- 
nary conditioning. 

From the foregoing account of types of responses occurring 
within the classical conditioning experiment, including redinte- 
grative responses, fractional component responses, prepara- 
tory responses and responses not classifying strictly as condi- 
tioning (including pseudo-conditioning and sensitization) it is 
evident enough that there is a rich diversity. The range of 
behavioral changes within experiments similar to Pavlov’s 
means that learning in these experiments has not been re- 
duced to the level of simplicity sometimes attributed to con- 
ditioned response learning. The fact that the behavioral 
changes range widely does not destroy the usefulness of Pav- 
lov’s experiment as a convenient reference situation; it means, 
however, that several alternative theoretical interpretations 
are possible, even within the somewhat restricted sphere of 
these experiments. 


Some Relationships within Classical 
Conditioning as Described by Pavlov 


The ordering of stimuli in experiments on simple condi- 
tioning, and the arrangements of discrimination experiments, 
have given rise to a number of terms descriptive of these rela- 
tionships as they were used in Pavlov’s laboratory, and these 
terms are now rather generally used by those who do condi- 
tioning experiments. In order to introduce these terms in the 
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context in which they were first used, some of the chief rela- 
tionships will be summarized here, without any effort to re- 
port the supporting quantitative evidence. Many of the situa- 
tions will be discussed again in later chapters, when they will 
be examined more critically. 

Temporal Relationships within Conditioning. In typ- 
ical experiments, the conditioned stimulus begins a few sec- 
onds before the unconditioned stimulus, and the two stimuli 
overlap in time. With different temporal relations the result- 
ing conditioned response shows certain characteristics which 
Pavlov has summarized as follows: 


A. Conditioned and unconditioned stimuli overlap tempo- 
rally. 

1. Simultaneous conditioned response. The conditioned 
stimulus begins from a fraction of a second to 5 sec- 
onds before the unconditioned stimulus, and continues 
until the latter occurs. The conditioned response tends 
to follow the beginning of the conditioned stimulus al- 
most immediately. 

2. Delayed conditioned response. The conditioned stimu- 
lus begins from 5 seconds to several minutes before 
the unconditioned stimulus, and continues until the lat- 
ter occurs. Although the conditioned response begins 
before the unconditioned stimulus, it follows the onset 
of the conditioned stimulus by a delay proportional 
to the length of the interval between the two stimuli. 
Delayed conditioned responses are difficult to form un- 


less a simultaneous conditi 
itioned response has alread 
been established. i d 


B. Conditioned and unconditioned stimuli 
temporally. 
3- Short-trace conditioned response. The conditioned 


stimulus is removed for a few seconds before the un- 
conditioned stimulus begins. 


4. Long-trace conditioned response. The 
E the cessation of the conditioned s 
the beginning of the unconditioned stimu 


do not overlap 


interval be- 
timulus and 
lus is I min- 
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ute or more. The conditioned response does not begin 
at the onset of the conditioned stimulus, nor at its 
cessation, but after an interval proportional to the 
time elapsing before the presentation of the uncondi- 
tioned stimulus. Long-trace conditioned responses are 
formed with greater difficulty than delayed conditioned 
responses. 

5. Backward conditioned response. The conditioned 
stimulus does not begin to act until after the cessation 
of the unconditioned stimulus. 


C. Time interval functions as the conditioned stimulus. 

6. Temporal conditioned response. An unconditioned 
stimulus is presented at regular intervals of time. If 
it is now omitted, a conditioned response will occur at 
approximately the usual interval. Intervals as long as 
30 minutes have been used successfully with dogs. 


Inhibitory Characteristics of Conditioned Responses. 
When the conditioned stimulus is followed by the uncondi- 
tioned stimulus at an effective interval, it is said to be rein- 
forced. Repetition of reinforcement is thus the essential pro- 
cedure for establishing and strengthening a conditioned re- 
sponse. Once the conditioned response has been established, 
repetition of the conditioned stimulus without reinforcement 
results in a gradual inhibition of response. The procedure is 
known as experimental extinction. In a typical experiment, as 
described by Pavlov, a conditioned salivary response in a dog 
might be extinguished by the presentation of from § to 20 
applications of the conditioned stimulus without the subse- 
quent feeding. After an interval of hours following extinction 
the conditioned response can again be elicited. This return of 
excitability, without further training, is known as spontaneous 
recovery. The sudden presentation of a novel, distracting 
stimulus may result in a brief temporary restoration of an 
extinguished response. This restoration is known as disinhibt- 
tion. The same kind of extra stimulus presented when the 
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conditioned response is well established may cause a tem- 
porary reduction or abolition of the response, called external 
inhibition. 

After a conditioned response has been developed to one 
stimulus, other similar stimuli may elicit the response without 
any specific reinforcement. This fact is known as generaliza- 
tion. Following such generalization it is possible to secure 
discrimination, which means that the individual responds to 
one stimulus but not to another stimulus which at some pre- 
vious time did elicit the response. Discrimination may be ac- 
complished in part through continued reinforcement of one 
stimulus, but more satisfactorily by reinforcing one of a pair 
of similar stimuli and never reinforcing the other. The latter 
procedure is known as the method of contrasts, and is used to 
determine the limits of sensory discrimination. Thus by re- 
inforcing a tone of 800 cycles and not reinforcing one of 812 
cycles, it has been shown that a dog can discriminate between 
the two tones because it reacts differentially to them. The 
neural process underlying generalization was called irradia- 
tion by Pavlov, and the neural counterpart of discrimination 
was called concentration. Since irradiation and concentration 
refer to hypothetical cortical processes, they should not be 
used as behavioral terms unless qualified to prevent misunder- 
standing. 

The term inhibition is used to denote a decrease or absence 
of a response which is the result of some form of positive 
stimulation. The ‘decrease resulting: from an extraneous cime 
ulus is called external inhibition, while the decrease resulting 
mic ened cas tg deed nant 
tion also in the sense of a nae Tee ae ae 
eS eee eee ee or mechanism under- 
confusion has unfortunatel ee ees Mue 

y resulted from this dual use of 
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the term. Inhibition will refer here only to the experimentally 
measured decrement in strength of response. 

A stimulus whose conditioned response has been inhibited 
by extinction or differentiation is called a conditioned inhibi- 
tory stimulus because of its functional properties. It does not 
itself elicit any observable response, but it is capable of pro- 
ducing an inhibition of the response to another stimulus with 
which it is presented. For example, Pavlov (1927, p. 77) de- 
scribes an experiment in which a whistle stimulus to which the 
salivary response had been extinguished, produced a decrease 
in the salivation to a tactile stimulus from 8 drops to 1 drop on 
the first occasion on which the two stimuli were presented 
together. 

The types of behavioral situation to which the different 
varieties of inhibition apply were classified by Pavlov as 
follows: 


A. External inhibition. p 
Temporary decrement of a conditioned response due 


to an extraneous stimulus. 


B. Internal inhibition. 
Internal inhibition develops slowly and progressively, 
when a conditioned stimulus is repeatedly presented 
under one of the following conditions: - 

1. Experimental extinction. The weakening of response 
to a conditioned stimulus which is repeated a number 
of times without reinforcement. 

2. Differential inhibition. A conditioned response given 
originally to either of two stimuli is restricted to one 
of them through the reinforcement of one and the non- 
reinforcement of the other. The non-reinforced nega- 
tive stimulus becomes inhibitory. 

3. Conditioned inhibition. A combination of stimuli is 
rendered ineffective through non-reinforcement, al- 
though the combination includes a stimulus which 
alone continues to evoke the conditioned response. ‘The 
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other stimuli in the combination are conditioned inhibi- 
tors. 


. Inhibition of delay. If a regular interval of sufficient 


duration elapses between the commencement of a con- 
ditioned stimulus and its reinforcement, during the 
early portion of its isolated action the conditioned 
stimulus becomes not only ineffective, but actively in- 
hibitory of other intercurrent activities. (There may be 
temporary disinhibition at the onset of the conditioned 
stimulus, so that there is a slight conditioned response 
before the inhibition is manifested.) 


C. Disinhibition. 


Temporary increment of an inhibited conditioned re- 
sponse due to an extraneous stimulus. This may be con- 


sidered as an external inhibition of an internal inhibi- 
tion. 


This brief summary of the principal descriptive character- 


istics of conditioning may serve as an introduction to the 
more detailed consideration of the phenomena in later chap- 
ters. Conditioning is defined as an experimental procedure for 
the establishment of simple associative connections between 
definite stimuli and responses. The Pavlovian reference ex- 
periment represents one procedure which has been exten- 


sively used. The other training methods will be discussed in 
the next chapter. 


Notes 


The terminology adopted for the 
description of classical conditioning 
is that of Pavlov’s translators, with 
a few minor alterations such as the 
substitution of the word response for 
reflex. Pavlov’s terms are already so 
familiar, that, in spite of a certain 
awkwardness, it is more serviceable 
to accept them than to substitute 


the terminologies proposed by other 
writers, none of which has won wide 
acceptance. The adoption of condi- 
tioned response instead of condi- 
tioned reflex follows the practice 
common among psychologists, a 
practice based on the desire to avoid 
the identification of the behavior 
with the spinal reflexes of Sherring- 
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ton. Some of the more common al- 
ternatives to Pavlov’s terms are 
given in the Glossary. 


Organisms and Responses Con- 
ditioned 


For a catalog of organisms and re- 
sponses conditioned, the bibliogra- 
phy of Razran (1937) is most help- 
ful. Hull (1934c) describes a large 
number of specific experiments in- 
volving different stimuli. His chap- 
ter is probably the best source for 
gaining acquaintance with the con- 
crete problems confronting those 
who perform conditioned response 
experiments and with the techniques 
which have been used. Razran’s 
reviews are helpful: animals other 
than dogs, Razran (1933a); chil- 
dren, Razran (1933b). Cf. also 
Wolff (1937). 


Apparatus and Technique 


Details regarding the construction 
of Pavlov’s sound-proof laboratory 
and the preparation of dogs for sali- 
vary experimentation are given by 
Podkopaev (1926). The laboratory 
care of dogs, and techniques used in 
motor conditioning, are described by 
Kellogg, R. C. Davis, and V. B. 
Scott (1939). A number of arrange- 
ments of apparatus for use with ani- 
mals and children are pictured by 
Razran (1933a, 1933b). Animal 
holders suitable for different species, 
and types of photokymographic 
equipment, are illustrated by Wendt 
and Dodge (1938). The most use- 
ful general treatment of recording 
equipment is by Wendt (1938). 
Other devices useful in connection 
with various problems of condition- 


ing are described by Brogden and 
Culler (1936), Culler, Finch and 
Girden (1934), Hanford and C. T. 
Morgan (1939), Loucks (1932, 
1934, 1938a), Shastin (1938). 


The Nature of the Condi- 


tioned Response 


The question of the descriptive 
similarities and differences between 
conditioned and unconditioned re- 
sponses is discussed more fully by 
Culler (1938b), Grant (1939b), Hil- 
gard (1936a, 1936b), Kellogg 
(1938a), Liddell, W. T. James, and 
O. D. Anderson (1934), Razran 
(19391), Schlosberg (1936, 1937), 
Warner (1932b), Woodworth 
(1938), Zener (1937). 

The modification called sensitiza- 
tion in the text was designated ‘al- 
pha’ conditioning by Hull (1934c) 
to distinguish it from the more usu- 
al conditioning (‘beta’). The ex- 
istence of ‘alpha’ conditioning was 
inferred from experiments by Wendt 
(1930) and Hilgard (1931). Later 
attempts to secure ‘alpha’ condi- 
tioning of eyelid reflexes have been 
unsuccessful (Bernstein, 1934, Hil- 
gard and Biel, 1937). Bernstein 
made the suggestion, concurred in 
by Hilgard and Biel, that alterations 
in set, not necessarily related to the 
precise repetition of conditioned and 
unconditioned stimuli, probably ac- 
count for whatever slight sensitiza- 
tion of the ‘alpha’ type that occurs. 
Hence, sensitization becomes a 
form of pseudo-conditioning. 

Although pseudo-conditioning 
does not qualify as a form of classi- 
cal conditioning, it may occur with- 
in classical conditioning experi- 
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ments, and hence, in its interaction 
with other processes, it may be im- 
portant as a factor in the observed 
modification (Harlow, 1939). The 
phenomenon is closely allied to the 
physiological principle of dominance 
described by Ukhtomski (1926, 
1938). According to this principle, 
any reaction which has been elicited 
repeatedly by strong stimulation 
may become dominant over other 
reactions, so that it becomes elicita- 
ble by a range of stimuli much wider 
than is normally the case. One illus- 
tration of the applicability of this 
principle to classical conditioning is 
found in the restoration of extin- 
guished conditioned responses. Fol- 
lowing extinction, the response to 
the conditioned stimulus is com- 
pletely restored by a few reinforce- 
ments. The restoration may be pro- 
duced, however, by presenting the 
unconditioned stimulus alone for 
several times. In this case, the 
principle illustrated is that of domi- 
nance, rather than of true recondi- 
tioning, for the restoration of sensi- 
tivity to the conditioned stimulus 
takes place without its combination 
with the unconditioned stimulus. 


Sensory Conditioning 


Association in its traditional 


forms was concerned with associa- 


tions among ideas and sensory proc- 
esses. While conditioning has tended 
to emphasize the association be- 
tween stimuli and movements, sev- 
eral investigators have attempted 
to apply the arrangements of the 
conditioning experiment to the in- 
vestigation of sensory processes. An 
unconditioned stimulus elicits a 
sensory process reported, for exam- 
ple, as the color red. If a tone is re- 
peatedly presented together with the 
unconditioned stimulus, will the 
tone come to elicit a corresponding 
experience of redness, i.e., a hallu- 
cinatory experience? The appearance 
of synesthesias in some individuals 
suggests a possible conditioned ori- 
gin. The findings of Kelly (1934) 
and of Cason (1936) were negative, 
although some success is reported 
by Bogoslovski (1937). Bogoslov- 
ski measured conditioning by the 
electrical sensitivity of the eye. A 
metronome combined with a visual 
stimulus was found after condition- 
ing to have raised the threshold of 
the dark adapted eye as measured 
by the amount of current which had 
to be passed through the eye to pro- 
duce a visual experience. Because of 
the fragmentary nature of the re- 
sults to date, these experiments were 
not included in the text as represent- 


ative of varieties of responses con- 
ditioned. 


Chapter 3 
INSTRUMENTAL CONDITIONING EXPERIMENTS 


HE METHODS of establishing simple conditioned responses 
T are not exhausted by the classical training procedure 
of Pavlov. Several other important procedures which have 
come to be known as conditioning will be described in this 
chapter. These training methods are distinguished from clas- 
sical conditioning by the fact that the reinforcement is pre- 
sented only after the conditioned response has been made. 
‘L When the occurrence of the reinforcement is contingent upon 
the organism’s behavior the procedure may be termed instru- 
mental conditioning, since the conditioned response is instru- 
mental in bringing about the reinforcement. Various types of 
reinforcement have been employed in instrumental condition- 
ing experiments, and illustrative examples of four types of 
procedure will be described below. These four are (1) reward 
training, in which the conditioned response is followed by the 
presentation of a positive incentive stimulus such as food, 
(2) escape training, in which the conditioned response is fol- 
lowed by the termination of a noxious stimulus such as elec- 
tric shock, (3) avoidance training, in which the conditioned 
response prevents the occurrence of a noxious stimulus, and 
(4) secondary reward training, in which the conditioned re- 
sponse is followed by a stimulus such as a token which has 
acquired reward value in previous learning- The differences 
will be more evident as the four varieties of instrumental con- 
ditioning are described in more detail. 

5I 
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Reward Training 


As a reference experiment for instrumental reward train- 
ing we may select a study by Grindley (1932), who taught 
guinea pigs to secure carrots by turning the head to one side 
at the sound of a buzzer. The relevant stimuli and responses 
were kept simple and made as nearly repetitive as possible. 
The guinea pig was held in a stand in a sound-proof room. 
Head movements were recorded kymographically and ob- 
served through a telescope from another room. A buzzer was 
sounded as the conditioned stimulus. If the animal turned its 
head to the right side while the buzzer sounded, a carrot was 
automatically presented on a lever in front of the guinea pig. 
After the animal had nibbled it, the carrot was removed by 
means of a string controlled by the experimenter. The ar- 
rangement differs from the Pavlov experiment in that the re- 
inforcement (carrot) is not presented at a fixed constant in- 
terval following the conditioned stimulus, but only after the 
animal has performed the head turning. In this procedure the 
conditioned response may properly be said to be instrumental 
in securing the reward. The head turning is not evoked by 
the reinforcing stimulus but must occur before the reinforce- 
ment is presented. 

Preliminary adaptation trials on 8 to 15 days were needed 
before the animals would eat promptly when food was pre- 
sented in the stand. When regular training began, learning 
was evidenced by a gradual decrease in the time between 
the onset of the buzzer and the appropriate head movement, 
a decrease from 100 seconds to 1 second. In the later trials, 
T pene i eee the animals often showed food re- 
eee : : g Movements and sniffing, which were 
aes einen ead movements to the right side. There 

ty iittle turning to the wrong side. Between trials some 
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of the animals were quiescent or asleep; others were active, 
moving the head or nibbling. 

It is evident that in an instrumental situation of this kind 
the measured conditioned response may not resemble at all 
the response to the reward stimulus. The sharp head move- 
ment to one side was not a part of the eating behavior, yet it 
was the response conspicuously strengthened. This reaction 
was selected out of the repertory of behavior which the ani- 
mal exhibited in the presence of the buzzer; behavior which 
included a slight startle, rigidity, widening of the eyes, rapid 
and regular respiration, struggling and investigatory re- 
sponses, as well as head movements. The responses to the 
buzzer were gradually reduced, except for the appropriate 
head movement rewarded by the carrot. 

Instrumental reward training has been studied extensively 
in many different animals and with various responses. It is 
not a new training procedure; it is the same as that described 
by C. L. Morgan in 1894 and designated a few years later 
by Thorndike (1898) as “trial, error, and accidental suc- 
cess.” Thorndike carried out a series of experiments in which 
cats were trained to escape from problem boxes to secure food 
outside. The boxes were constructed with sides of vertical slats 
so that the food was visible to the cat. A door in the box 
opened as soon as the cat performed a certain movement, such 
as pulling a string hanging from the top, or pressing a latch. 
A hungry cat when first placed in such a box shows a con- 
tinued but variable activity, reaching between the slats toward 
the food, scratching at the sides, moving all about the box. 
In the course of this activity the cat eventually hits the re- 
lease mechanism and immediately secures the food. The first 
successful response appears to be largely a matter of chance. 
On successive trials the cat’s activity becomes concentrated 
in the region of the release mechanism and other activities 
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gradually drop out until eventually the animal performs the 
correct act as soon as it is placed in the box. 

The conception of an instrumental act must not be under- 
stood too narrowly. In the illustrations which have been 
given, what the organism does produces the reward in some 
mechanical fashion, either by bringing the organism into the 
presence of the reward through its own movements, or by 
bringing the reward to it through some release mechanism. 
The act which precedes the reward need not be mechanically 
related to the delivery of the reward, but instead may be 
purely arbitrary. For example, in one experiment Thorndike 
taught the cat to lick itself in order to be released from the 
cage. Training the dog to “shake hands” by rewarding is 
another illustration of an arbitrary act of this sort. The es- 
sential aspect of instrumental training is that the reward de- 
pends on what the organism does and follows the response 
in some systematic fashion. 

It is entirely a question of terminological definition whether 
instrumental reward training is to be considered a type of 
learning different from conditioning or as a separate type of 
conditioning. There is a certain convenience served by adopt- 
ing the latter point of view, and historical precedent for it 
is readily found. Students in Pavloy’s laboratory have de- 
scribed instrumental training in the terminology of condition- 
ing. For example, Ivanoy-Smolensky (1927c) carried out a 
series of experiments in which children were taught to obtain 
food or other rewards by pressing a bulb when certain con- 
ditioned stimuli were presented. S. Miller and Konorski 
(1928) designated as a second type of conditioning their ex- 
BG cio hy nk eae 
dog’s leg passively and ree i i er ee, 
dog soon learned to flex its le se i a a ontan he 
a a eee an ae ae at the signal. In some of their 

> er ones were performed in Pav- 
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lov’s laboratory, Konorski and Miller also measured saliva- 
tion, and hence secured classical and instrumental condition- 
ing at the same time (Figure 4). When hydrochloric acid was 






SATE irra 


Ficure 4 


Dracram oF ONE OF THE Konorski AND MILLER EXPERIMENTAL 
ÅRRANGEMENTS Usep IN Paviov’s LABORATORY 


a—system to measure dog’s salivation; b—system to measure the flexion 
of the dog’s leg; c—system to produce the passive flexion. After Konorski 
and S. Miller, from Razran (19391). 


substituted for food, however, the conditioned leg flexion 
was abolished, although conditioned salivation was main- 
tained at full strength. Instead of flexion to the conditioned 
stimulus, as with food reward, the leg was stiffened and could 
be flexed only by applying excessive force. 

The most extensive program of research on instrumental 
training has been carried on by Skinner (1938) who considers 
it a distinct type of conditioning. In his procedure a rat was 
placed in a sound-proofed box containing a small lever which 
when pressed, automatically released a pellet of food into a 
tray. When first placed in the box a hungry rat would move 
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about, exploring the various parts of the box and in the 
course of minutes chance to press the lever, after which it 
ate the food. The rat was not removed from the box as in 
the case of Thorndike’s situation, but was immediately free 
to press the lever again. Since the training procedure was 
not divided into discrete trials, the progress of learning was 
measured in terms of the rate of responding—the number of 
lever pressings per unit of time. It should also be noted that 
in this training procedure a conditioned stimulus was not pre- 
sented in the usual way. The complex of external and internal 
stimuli arising from the box and lever and from processes 
within the animal corresponded to a conditioned stimulus. 
Since these stimuli were continuously present, it was always 
possible for another conditioned response to be elicited as 
soon as the preceding one was terminated. 


Escape Training 


Instrumental conditioning can be carried out by employ- 
ing in place of a positive reward the cessation of a persisting 
disagreeable or noxious stimulus such as electric shock, im- 
mersion in water, or excessive temperature. This type of train- 
ing procedure has seldom been used in laboratory condition- 
ing experiments, although numerous examples are found in 
the learning of animals in their natural environment. An 
experiment reported by Mowrer (1940) may serve as the 
paradigm of instrumental escape training. A rat was placed 
in a box with a metal grille floor through which an electric 
shock could be administered to the animal. The shock was 
gradually increased during 2.25 minutes from zero to a stable 
intense value. At one end of the box was a pedal arrange- 
ment which would turn off the shock whenever pushed. The 
shock remained off as long as the pedal was depressed but 
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when the pedal was released the shock would automatically 
start to build up again. 

Rats in this situation began to show agitated behavior 
about 6o seconds after the beginning of the shock. As the 
shock became more intense the rats engaged in vigorous un- 
directed activity characterized by jumping, running, squeal- 
ing, biting at the grille and random thrashing about. In the 
course of this activity the rat always chanced to hit the pedal 
and thus terminated the shock. The pedal response occurred 
on the first trial only after from 3 to 6 minutes of activity, 
but by the tenth trial the rats pressed the pedal as soon as 
they began to feel the shock. 

Many of the essential features of escape training are similar 
to reward training. In both procedures the correct response 
is not elicited by the reinforcing stimulus (i.e., the reward or 
escape) but must occur spontaneously on the first trial before 
the reinforcement is presented. The nature of the conditioned 
response is therefore not determined by the reinforcement. 
Rats might be trained to terminate a shock by pulling a loop, 
pushing a hanging rod, sitting up on their hind legs, or in- 
deed by any response which is already in their repertory of 
behavior and likely to occur in the experimental situation. 
Perhaps the most frequent example of escape learning is that 
in which the organism moves from a region of noxious stimu- 
lation to a neutral region. 

It should be carefully noted that instrumental escape train- 
ing is not the same sort of training as that in the typical pun- 
ishment situation. In the latter, shock is administered follow- 
ing incorrect response and may produce a decreased tendency 
to perform that response. In escape training, the shock is 
present before the conditioned response occurs, and escape 
from the shock produces an increased tendency to perform 
the response which immediately preceded the escape. 
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Avoidance Training 


Differing from both instrumental reward and instrumental 
escape is that variety of training procedure in which the 
learned reaction prevents the appearance of a noxious stim- 
ulus. The reference experiment is the classical one by Bekh- 
terev on conditioned withdrawal responses of hand or foot, 
in which the electric shock may be avoided by the performance 
of the conditioned response. If the electrodes are attached 
to the subject and the shock is delivered on every trial, the 
experiment is reduced to the classical Pavlov procedure. In 
some cases results appear to be very similar whether or not 
the conditioned response prevents the shock (Schlosberg, 
1934, 1936; Hilden, 1937; Munn, 1939). In other cases 
there is a distinct difference. This is well shown in an ex- 
periment performed by Brogden, Lipman and Culler (1938). 
A guinea pig was placed in a revolving cage and after a con- 
ditioned stimulus (buzzer) it was given a shock which evoked 
running behavior. One group of animals, trained according 
to the Pavlov procedure, were shocked whether or not they 
ran. Another group, trained according to the arrangements 
of avoidance learning, were not shocked if they ran. Learning 
began similarly in both groups, but reached a much higher 
level of performance in the second group. The results are 
plotted in Figure 5. Guinea pigs which were shocked whether 
or not they ran continued to show anticipatory agitation at 
the sound of the buzzer, but after the first few trials the tend- 
ency to run did not increase; those shocked only if they 
did not run developed the habit of running promptly at the 
sound of the buzzer. Learning in this situation appears to be 
ace a Sa sede on eni of the shock. It differs 
the conditioned Eee. i i jae ae eee ins ae 
change—food or the cessation a a pero Me 
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Comparison OF LEARNING PERFORMANCE WITH INSTRUMENTAL AVOIDANCE 
TRAINING AND CLASSICAL CONDITIONING 


The solid line represents the frequency of conditioned responses (running) 
of 4 guinea pigs during the two seconds that a buzzer sounds, when running 
prevents the appearance of a shock which occurs on those trials in which 
the animal does not run. The broken line represents the corresponding values 
for another group of 4 animals when the buzzer is invariably followed by 
shock whether or not the animal runs. The greater learning under circum- 
stances of shock-avoidance than of regular shock indicates that the shock 
is not itself to be considered the reinforcement. (Brogden, Lipman, and Culler, 


1938.) 


ance training the new response is strengthened in the absence 
of any such stimulus; indeed, it is strengthened because of 
the absence of such a stimulus. Absence of stimulation can 
obviously have an influence on behavior only if there exists 
some sort of preparation for or expectation of the stimula- 
tion. This strengthening through absence of stimulation is im- 
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portant enough to make it desirable to consider avoidance 
training separately from the other kinds of instrumental train- 
ing. 

Since the organism must receive a certain number of shocks 
before it learns to avoid the shock following the conditioned 
stimulus, the first few trials of instrumental avoidance train- 
ing are identical in procedure with classical conditioning. This 
is clear in the results shown in Figure 5 and is further il- 
lustrated in experiments on dogs by Culler, Finch, Girden and 
Brogden (1935). The conditioned stimulus was a tone which 
was followed by a shock to one paw whenever the dog failed 
to withdraw its leg. On the first few trials the dog received 
a shock every time. A conditioned response to the tone rapidly 
developed which was at first a “reduced replica” of the original 
reaction to the shock, and included generalized struggling 
movements of the body as well as retraction of the leg. In 
the course of further training trials the conditioned response 
became restricted to a precise and adaptive flexion of one leg 
by which the shock was avoided. Thus the experiment pro- 
gressed from a preliminary stage of classical conditioning to 
a later stage of instrumental avoidance training. The similar- 
ity to the eyelid series presented in Figure 3 (Chapter 2, p. 38) 
may be noted. 

It is apparent that the shock cannot be considered an un- 
conditioned stimulus during the later stages of avoidance 
training. Omission of the shock on trials in which the condi- 
tioned flexion occurs does not produce extinction but instead 
strengthens the conditioned response. Extinction can be se- 
cured only if the dog fails to respond for some reason and 
thereby “discovers” that the tone is no longer followed by 
shock. After several trials of this sort extinction progresses 
o nant 

experiments by Finch and 
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Culler (1934), who showed that the shock need not be de- 
livered to the leg. After the conditioned withdrawal response 
had once been firmly established, the shock to the leg was 
disconnected, and a shock to the chest was administered when- 
ever the dog failed to react to the tone. No extinction oc- 
curred and the conditioned response could be maintained at 
maximum strength for indefinite periods under these condi- 
tions. It was observed that the shock to the chest never itself 
evoked the leg withdrawal response. 

An important clue to the understanding of instrumental 
avoidance learning is provided in a further experiment by 
Finch and Culler (1935). A buzzer was connected with the 
shock circuit, so that the buzzer would sound whenever the 
shock was turned on. Conditioned leg withdrawal avoided the 
shock, but the buzzer continued to sound. With conditioning 
well established extinction was undertaken, in one case by 
omitting both the shock and the buzzer following the tone, in 
the other case by omitting the shock but continuing the buzzer. 
Three dogs, each extinguished under both conditions, showed 
a mean time of extinction without the buzzer of 100 trials, 
with the buzzer of 372 trials. The difference, which is statisti- 
cally significant, must be attributed to the reinforcing effect 
of the buzzer. The buzzer, however, cannot be considered an 
unconditioned stimulus for it did not by itself elicit leg with- 
drawal even after its repeated combination with the shock. 


Far from being a stimulus to foot-withdrawal, the buzzer 
becomes the cue to foot-replacement. The buzzer, when it 
stops sounding, makes known that the trial is over and that 
the grid again is safe. Just as the tone announces that the grid 
will soon be ‘hot’ so the buzzer announces that it will soon be 
‘cold.’ 

By omitting the shock and continuing the buzzer, which 
has throughout the training period been coterminous with it, 
we are in a way tricking or deceiving the animal. The buzzer 
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has come to symbolize charge-on-the-grid; it continues now 
to do so, even though it has become a false symbol. (Finch 
and Culler, 1935, p. 659.) 


The buzzer is not a usual reinforcing agent. It derives what- 
ever reinforcing significance it has from its association with 
shock, so that it maintains the animal’s avoidance reactions 
originally based on shock. The stimuli which serve the rôle 
of secondary or derivative reinforcing agents in ordinary 
avoidance training are not always as apparent as the buzzer 
in this experiment. In the absence of the buzzer, the sight of 
the shock electrodes, the general experimental situation, and 
the perception of the time interval following the conditioned 
stimulus may operate in the same manner. 


Secondary Reward Training 


In avoidance training, as has been indicated, some feature 
of the experimental situation other than the punishment it- 
self becomes capable of reinforcement as a result of its asso- 
ciation with punishment. There is a parallel case in certain 
experiments with rewards, in which a neutral stimulus object 
may acquire the reinforcing properties of a reward if it has 
been repeatedly associated with a primary reward. The prob- 
lem of explanation is essentially the same in avoidance learn- 
ing and in secondary reward learning: how indifferent stimuli 
come to reinforce responses appropriate to punishment or to 
reward. Because of this common problem of explanation, as 
a representative experiment secondary reward training be- 
longs logically with avoidance training rather than with simple 
instrumental reward and escape. 

Illustrative of this type of training is Wolfe’s (1936) study 
of token incentives in the behavior of chimpanzees. The tokens 
were small disks similar to poker chips to which the animals 
were originally quite indifferent. Preliminary training con- 
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sisted of teaching the chimpanzees to insert tokens in the slot 
of a vending apparatus which automatically released a grape 
to the animal. Differential reward values were established for 
different colored tokens. For example, a blue token secured one 
grape, a red token, two grapes, and a brass token no food at 
all. From a heterogeneous collection of tokens a trained 
chimpanzee would first select the red tokens for deposit in 
the vending apparatus, and then the blue ones, leaving the 
brass untouched. 

After having learned the reward value of tokens, chimpan- 
zees were trained to perform simple instrumental responses 
to secure the tokens. One task required the lifting of a lever; 
another involved pulling in a small sliding tray by means of 
a cord. The animals would continue to work at these tasks for 
a number of tokens which could not be exchanged for food 
until later. In an extension of these experiments, Cowles 
(1937) found that token rewards were effective as incentives 
for the acquisition of simple position habits, of complex five- 
choice position habits, of visual size and color pattern dis- 
criminations, and of delayed responses. A position habit re- 
quiring as many as 20 trials could be completely learned in 
one session with tokens as the sole reinforcing agent and 
therefore prior to any reception of food reward. 

Secondary rewards such as approval, money, prestige and 
so forth are of unquestioned importance in directing most 
learning by adult human individuals. In customary experi- 
ments with college students on the learning of rote verbal ma- 
terial, mirror drawing, or code substitution the only reward 
necessary is the knowledge of which responses are correct 
and which incorrect. 

The mechanism of secondary reward training is difficult to 
analyze in the case of human learning because of the com- 
plexities introduced by the use of verbal language. The es- 
sential features of the procedure have, however, been success- 
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fully demonstrated with animals such as rats, cats and dogs 
as well as with chimpanzees. The results in general indicate 
that a secondary reward is only slightly less effective in the 
establishment of new learned responses than the primary re- 
ward with which it has been associated. The secondary reward 
loses its reinforcing value, however, if it is not itself repeatedly 
followed by the primary reward. In the chimpanzee experi- 
ments it was found that a token which was immediately ex- 
changeable for food was more effective as an incentive than 
tokens which were exchangeable only at the end of a series 
of trials. And if the secondary reward was frequently pre- 
sented without the subsequent primary reward the response 
became extinguished and its incentive value disappeared. 

From a consideration of these characteristics of secondary 
rewards it is apparent that in the typical food-reward training 
situation, any stimulus occurring after the conditioned re- 
sponse and prior to the securing of food may take on the 
properties of a secondary reward in the course of training. 
Because of its position in the sequence of activities, such a 
stimulus has sometimes been designated a sub-goal. The sight 
of food or of the food container thus becomes an incentive in 
itself. Grindley (1929) studied the behavior of chickens which 
were placed in a narrow runway at the other end of which 
were grains of rice under a piece of glass, such that the chick- 
ens could see the rice but could not reach it. During the first 
4 or 5 trials there was a gradual increase in running speed 
down the alley toward the food, indicating that the sight of 
food induced learning even when the eating was prevented. 
On subsequent trials the running speed became progressively 
slower. 

The reinforcing action of a sub-goal in the learning by rats 
to press a lever to secure food has been clearly demonstrated 
by Skinner (1938) and by Bugelski (1938). The apparatus 
used was so constructed that when the lever was pressed there 
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was a clearly audible click occasioned by the release of a pellet 
of food into the food receptacle. After the habit had been 
firmly established extinction was carried out in two ways 
(Bugelski, 1938). For half of the rats, pressing the bar re- 
sulted in the click but no food was delivered; for the other 
half the response was not followed by either the click or the 
food. The former group made 30 percent more responses 
during extinction than the latter group, indicating that the 
click had maintained the conditioned response for a short 
time until its reinforcing value was itself extinguished by the 
absence of subsequent food. The function of the click here 
parallels very closely that of the buzzer in the experiment of 
Finch and Culler (1935) described on p. 61. An explanation of 
the reinforcing effect of sub-goals will therefore go far toward 
an explanation of avoidance learning. Attempted explanations 
are postponed until Chapter 4. 


The Nature of Conditioned 
Instrumental Responses 


Since in all types of instrumental learning the response 
which is strengthened may be entirely unlike the response to 
the reinforcing stimulus, the origin of the conditioned response 
is not accounted for by the nature of the reinforcement. The 
training procedure itself is such that the conditioned response 
must occur before it can be reinforced. The conditioned re- 
sponse is therefore already in the behavior repertory of the 
subject, and the learning process consists only in strengthen- 
ing the response so that it occurs promptly and with de- 
pendable frequency in the presence of the conditioned stimu- 
lus. The reinforcing stimulus differs from the unconditioned 
stimulus of the Pavlov procedure in that it can never be the 
original stimulus for the new response. The conditioned re- 
sponse first occurs as a random or spontaneous movement, 
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a movement selected under the influence of a drive, a pre- 
viously learned movement elicited as a generalized reaction 
to some feature of the environmental situation, or an ap- 
propriate movement which has been encouraged by some spe- 
cial means. The problem of the source of the conditioned 
response merits further consideration. 

Random or Spontaneous Movements. Trial-and-error 
theories imply that the original success occurs by chance. 
That is, the organism in a restless state varies its behavior 
throughout a wide range of movements which appear to be 
random or spontaneous because they cannot be accounted 
for by recognized stimuli. They are not random, however, in 
the sense that their nature is a matter of chance, and they 
are not spontaneous in the sense that antecedent conditions 
do not determine their occurrence. They are acts which are 
organized and released by the central nervous system inde- 
pendently of identifiable external stimuli, and might more 
appropriately be named self-initiated acts. Such movements 
have recently been given much emphasis by Skinner (1938), 
who calls them operants to distinguish them from respondents, 
which are movements elicited by recognized stimuli. 

The behavior of an animal in a new situation is not entirely 
random and spontaneous. Although the activity may not be 
directed specifically toward the correct solution of the prob- 
lem, it is not completely undirected. The cats in Thorndike’s 
puzzle box spent much time on the first trial reaching through 
the slats toward the food, and the rats in Skinner’s situation 
explored the corners and projecting portions of the box. In 
some cases, as indicated below, it is possible to stipulate fairly 
definitely the original stimulus which occasions the conditioned 
response. 

} Responses Determined by the Drive. The problem situa- 
tion is characterized by the presence of a drive which increases 
the restless behavior. Since drives have been associated in 
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the animal’s past with certain acts leading to reward, the re- 
sponses likely to appear in a new experimental situation may 
include those which have occurred previously while the par- 
ticular drive was active. Several recent experiments have 
demonstrated that animals can discriminate among their own 
drives; i.e., that a drive may be a conditioned stimulus evok- 
ing specific behavior (Hull, 1933, Leeper, 1935). 

Generalized Responses to External Stimuli. When an 
organism is in a novel situation, it reacts to features of the en- 
vironment bearing some marks of familiarity. In condition- 
ing terms this is known as generalization; that is, previously 
conditioned responses are evoked by stimuli similar to the 
original learning situation. A rat which has secured food by 
digging in one box with a sawdust floor is more likely to be- 
gin digging when in a new box with a sawdust floor than is 
an animal without this previous experience. 

Special Training. The appropriate response in instrumen- 
tal learning may be encouraged in many artificial ways, de- 
pending upon the ingenuity of the experimenter or animal 
trainer. The response may be suggested by passive move- 
ment. This is common practice in animal training, as in teach- 
ing a dog to “shake hands.” Or the leg movement which is 
rewarded may be originally produced by shock to the foot. 
Lever-pressing is sometimes enhanced by placing a bit of 
food on the lever to direct the animal’s exploratory activity to 
that region. String-pulling behavior is facilitated by smearing 
food on the string, jiggling the string, and so on (T. L. Mc- 
Culloch, 1934). Mechanically restricting the possibilities of 
movement, as in Grindley’s experiment, is another method 
of assuring the appearance of the appropriate reaction. The 
experiment proper consists in the strengthening of the re- 
sponse after it has once appeared. This is an important dif- 
ference from Pavlov’s experiment. The difference is illus- 
trated by the question: How many trials are needed before 
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the first conditioned response appears? This may be asked 
with respect to Pavlov’s experiment, but is meaningless with 
respect to instrumental training in which the conditioned re- 
sponse must always occur before the first trial is completed. 


Comparison of Classical and 
Instrumental Conditioning 


Classical and instrumental conditioning are similar in 
many basic respects and different in others. All conditioning 
experiments involve the presentation of a conditioned stimulus 
together with a stimulus situation which originally evokes a 
response, and with such additional features as will produce 
a strengthening of the new response to the conditioned stimu- 
lus. The various types of training will be examined with re- 
spect to these characteristics. 

Conditioned Stimulus. In many experiments there is no 
specific conditioned stimulus which is turned on and off un- 
der the control of the experimenter. In these instances, the 
total situation confronting the subject may be considered as 
the analogue of the conditioned stimulus. The particular as- 
pects of the situation to which the subject is responding can 
be discovered only by experimental variation of the situation. 
In this respect there are no differences between the classical 
and instrumental training procedures. Examples have been 
given of the use of a definite conditioned stimulus in both 
types; other examples are readily found in which the total 
situation served as a conditioned stimulus. Kleitman and 
Crisler (1927) found that the conditioned salivation based 
on administration of morphine was evoked when the dogs 
were brought to the experimental room and placed in the re- 
straining stand. Skinner’s instrumental reward training con- 
tained no conditioned stimulus; the rats pushed the lever to 
obtain food whenever they were in the experimental box. 
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This procedure was modified in some instances so that the 
conditioned response was evoked only when a light in the 
box was turned on (Skinner, 1935a). Measures of latency and 
of the effect of changes in the intensity of stimulus can of 
course be made only when the conditioned stimulus is a con- 
trolled feature of the training situation. 

Original Stimulus for the Conditioned Response. Be- 
fore a response can become associated with the conditioned 
stimulus, the response must be made to occur. The various 
training procedures employ different means for the original 
elicitation of the response to be conditioned. In classical condi- 
tioning an unconditioned stimulus is presented which invari- 
ably evokes the response. In instrumental training of all types 
the experimenter depends upon the occurrence of the response 
in the course of random spontaneous activity, although the 
chances of occurrence are increased if the subject has pre- 
viously learned to perform the response under similar con- 
ditions of drive or of external stimulation. The response which 
occurs is then punished or rewarded, but the punishment or 
reward, unlike an unconditioned stimulus, does not produce 
the response in the first place. 

Reinforcement. That feature of the training situation 
which is necessary for the strengthening of the new response 
to the conditioned stimulus is designated as the reinforcement. 
The reinforcement is identified by the fact that when it is 
present on successive training trials the conditioned response 
increases in strength; when it is omitted, the conditioned re- 
sponse decreases in strength; i.e., it is extinguished. While 
the nature of the conditioned response is determined by its 
original stimulus, its strength is determined by the intensity 
and the number of repetitions of the reinforcement. The par- 
ticular reinforcement operating in any training situation is 
discovered experimentally in terms of its influence upon the 
strength of the conditioned response. This definition is cir- 
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cular; the following chapter presents a discussion of the pos- 
sibilities of defining reinforcement independently of its 
strengthening effect on conditioned responses. 

There are important differences in the nature of the rein- 
forcement operating in the various types of training. In the 
classical situation the activity produced by the unconditioned 
stimulus supplies the reinforcement, and if the unconditioned 
stimulus is omitted on a series of trials the conditioned re- 
sponse undergoes extinction. It is important to note that the 
unconditioned stimulus is both the original stimulus determin- 
ing the form of the conditioned response, and the source of 
the reinforcement. In instrumental training the reinforcement 
occurs by way of a reward, an escape, or an avoidance of pun- 
ishment. In this situation the original stimulus determining the 
nature of the conditioned response is entirely distinct from 
the reinforcement. 

The difference in reinforcement among the various types of 
training may be described in another way. If the conditioned 
response resembles the behavior evoked by the reinforcing 
agent, the reinforcement may be spoken of as homogeneous. 
This is the typical observation of classical conditioning. In 
instrumental reward and escape training the conditioned re- 
sponse bears no essential resemblance to the behavior evoked 
by the reinforcing agent, and the reinforcement may be desig- 
nated as heterogeneous. Manipulating the latch in Abe Thorn- 
dike problem box, for example, has nothing in common with 
the response of eating food. If this terminology is extended to 
instrumental training based on avoidance of punishment or 
on secondary reward, the reinforcement may be designated 
as derived. In these cases it is clear that the conditioned re- 
sponse is not strengthened by the punishment or the primary 
reward, but by substitutes for them. The reinforcement is 
ie ae ee process within the subject instigated 

pends for its reinforcing power upon 
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the previously experienced punishments and rewards. The 
designation ‘derived reinforcement’ has been chosen to imply 
the dependent nature of the reinforcement in avoidance and 
secondary reward training. 

The differences among the several types of conditioning pro- 
cedure with respect to the original stimulus for the conditioned 
response and with respect to the nature of the reinforcement 
demand more extended consideration. In the following chap- 
ter the analysis of this problem will be carried further in or- 
der to see whether a single set of principles can successfully 
account for the variety of relationships which have been 


pointed out. 


Notes 


Types of Conditioning Pro- 
cedure 


The suggestion that instrumental 
conditioning be considered a type 
distinct from classical conditioning 
was made by S. Miller and Konor- 
ski (1928), and later by Skinner 
(1935b). The differences in their 
viewpoints have been discussed in 
several papers (Konorski and S. 


Miller, 1937a, 1937b; Skinner, 
1937). Two-fold classifications of 
learning situations, corresponding 


roughly to the distinction between 
classical conditioning and instru- 
mental conditioning, have been pro- 
posed by other writers. Thorndike’s 
distinction between associative shift- 
ing and learning by trial-and-error 
(1911, 1935) is of this sort, as is also 
Finch and Culler’s (1935) differen- 
tiation between ordinary condition- 
ing and motivated learning. Schlos- 
berg (1937) distinguishes between 


conditioning and success learning. 
Razran (1939i) redefines the prob- 
lem in terms of differences between 
specific or quantitative conditioning 
and more general or qualitative con- 
ditioning. He believes that ordinary 
conditioning is primarily of the spe- 
cific sort, while learning under the 
‘law of effect’ is characterized by 
the conditioning of “a general or- 
ganismic adient quality or tend- 
ency.” 

The different varieties of training 
do not represent pure types of learn- 
ing requiring necessarily different 
principles of explanation. This is 
evident from the interpenetration 
of the principles used to account for 
various forms of learning as ana- 
lyzed by Dashiell (1935), Kellogg 
(1938b), Tolman (1934). The vari- 
ous types of conditioning differ in 
important aspects, however, which 
explanatory theories must take into 
consideration. 
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Skinner’s Type S and Type R 


Skinner’s distinction between two 
types of conditioning, mentioned 
above, has been made by him the 
basis for an extended treatment of 
behavioral laws differing for each 
type (1938). Since his types corre- 
spond roughly to our distinctions in 
Chapters 2 and 3 between classical 
and instrumental conditioning, the 
differences between his classifica- 
tions and ours should be clarified in 
order to avoid misunderstanding. 
He designates Pavlov’s experiment 
Type S, since the conditioned stim- 
ulus is important as an experimental 
variable, and he calls the instru- 
mental variety Type R, since it is 
the conditioned response, rather 
than its stimulus, which enters as 
the significant variable. This dis- 
tinction is based in turn on the 
difference between respondent and 
operant behavior. Respondent be- 
havior, like ordinary spinal reflexes, 
is elicited by specified stimuli. In 
Type S it is respondent behavior 
which is conditioned. Operant be- 
havior, on the other hand, is emitted 
by the organism, and its stimuli can- 
not be specified. In Type R, operant 
behavior is conditioned. In situations 
involving discrimination operant be- 
havior may become correlated with 
a differential cue, but such a cue is 
not to be confused with an eliciting 
stimulus. When an operant response 
becomes correlated in this way with 
a stimulus, so that the response ap- 
pears superficially to be elicited by 
the stimulus, the relationship is that 
of a ‘pseudo-reflex.’ The laws of re- 


spondent behavior do not apply to 
pseudo-reflexes. 


Confusion arises because Skinner 
appears to fit actual experiments 
into types based on abstract fea- 
tures of the experiment. No actual 
experiment is a pure case. Skinner’s 
types correspond more nearly to our 
distinctions between homogeneous 
and heterogeneous reinforcement 
than to our classifications of experi- 
ments. Type S really represents that 
abstracted feature of Pavlov’s ex- 
periment which may be described in 
terms of homogeneous reinforce- 
ment; Type R represents that aspect 
of reward training which can be de- 
scribed in terms of heterogeneous 
reinforcement. Skinner recognizes 
that Pavlov’s experiment is not a 
pure case of Type S, but strictly 
speaking must be formulated as a 
pseudo-reflex (1935b, p. 73). His 
own experiment represents a com- 
plicated chain of events (1938, pp. 
52-55), from which the principles of 
Type R must be abstracted by ap- 
propriate analysis. 

The classifications in Chapters 2 
and 3 are based on what happens in 
representative experiments, so that 
the behavioral changes included are 
broader than Skinner’s abstracted 
Types S and R. It was recognized in 
Chapter 2 that classical condition- 
ing experiments are much more com- 
plicated than Type S suggests, and 
the various types of response (red- 
integrative, fractional component, 
preparatory) were summarized. Sim- 
ilarly, the examples of instrumental 
learning in Chapter 3 are not pure 
cases of Type R. The example of re- 
ward training chosen from Grind- 
ley involved a discriminated condi- 
tioned stimulus (buzzer) which 
was the occasion for the operant 
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(head-turning) leading to the re- 
ward (carrot). This is a discrimi- 
nated operant according to Skin- 
ner, and is classified as pseudo-reflex. 
Therefore classical conditioning is 
not limited to Type S and instru- 
mental conditioning is not confined 
to Type R. The task of coordinating 
empirical data from a wide variety 
of situations made it evident that a 
classification based on experimental 
arrangements was more feasible 
than a classification based on logical 
abstractions or idealized experi- 
ments. While for our purposes we 
have rejected Skinner’s classification 
of conditioning types, we shall have 
further occasion to discuss aspects 
of his classification useful in under- 
standing processes inferred to be 
going on during conditioning. 
Because of the prominence in our 
discussion of what Skinner would call 
discriminated operants or pseudo- 
reflexes, a comment is in order re- 
garding our use of the word ‘stimu- 
lus.” It must be clear from context 
that we refer to a stimulus as an in- 
stigator in the sense familiar in the 
psychological laboratory, as, for ex- 
ample, in the conventional reaction- 
time experiment. In this experiment, 
the stimulus is that event from 
which the latent time of the re- 
sponse is measured. It is in Skinner’s 
terminology a discriminated stimu- 
lus, and the response is a discrimi- 
nated operant. Most of the correla- 
tions between stimuli and responses 
with which we deal are of this kind. 
While responses are said to be 
evoked or elicited, this is a matter of 
convenience, and no more is intend- 
ed than that the stimulus is in some 
sense the occasion for the response. 


Reward Training 


Rewarded learning has often been 
studied in conditioning laborato- 
ries, e.g, Eroféeva (1916), Beritov 
(1932), Dworkin (1934a), Wendt 
(1934). The lever-pressing situation 
introduced by Skinner and exten- 
sively studied by him has also been 
used by a number of workers in 
other laboratories who are quanti- 
fying the relationships within’ re- 
warded learning which correspond to 
the descriptive characteristics of 
classical conditioning, e.g., Arake- 
lian (1939), Ellson (1938), S. B. 
Williams (1938), Youtz (1938a, 
1938b, 1939). 


Escape Training 


Termination or reduction of a con- 
tinuous noxious stimulus has been 
employed more frequently in estab- 
lishing complex habits than in con- 
ditioning. Muenzinger and Fletcher 
(1936) trained rats in a visual 
brightness discrimination by deliv- 
ering a continuous electric shock 
through the floor of the apparatus. 
By making the correct choice re- 
sponse the rats escaped into a goal 
box which was not electrified. Es- 
cape from water has been used as 
motivation for rats in maze learning 
by O. C. Glaser (1910), McDougall 
(1927), F. L. Ruch (1930) and 
Macfarlane (1930). 


Avoidance Training 


The distinction between classical 
conditioning based on noxious un- 
conditioned stimuli and instrumen- 
tal avoidance training has been 
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made by several writers. Wood- 
worth (1938, pp. 101-107), for ex- 
ample, divides experiments into 
three classes: those in which the 
conditioned response neither pro- 
duces nor avoids the unconditioned 
stimulus, those in which it produces 
the unconditioned stimulus, and 
those in which it escapes a noxious 
stimulus. He accepts the first two of 
these as conditioning, but rejects 
“stable and successful avoidance re- 
actions” on the basis of Schlosberg’s 
(1937) argument that the noxious 
stimulus is not really a reinforcing 
agent. If derived reinforcement is 
accepted, avoidance training may 
also be considered a variety of con- 
ditioning. 

Shock, or other noxious stimuli, 
may be used in a variety of training 
procedures which are not always 
clearly distinguished. The following 
list is a condensed summary of the 
more important usages. (1) Shock 
may be used as the unconditioned 
stimulus in classical conditioning. 
In this procedure the shock always 
follows the conditioned stimulus reg- 
ularly, whether or not any condi- 
tioned response has been made. (2) 
Shock may be terminated by the ap- 
propriate response, as in instrumen- 
tal escape training. (3) Shock may 
be given whenever the subject 
makes a particular response. This is 
the typical punishment situation, 
and leads to reduction in strength of 


the punished response but does not 
guarantee the positive learning of 
any other response. It is a method 
for the elimination rather than the 
acquisition of a response. (4) Shock 
may be prevented by an appropriate 
response, as in instrumental avoid- 
ance training. (5) Shock may be 
given in such a way that it provides 
no specific reinforcement for correct 
response but modifies the learning in 
some general way. Muenzinger and 
A. Wood (1935) found that shock 
for both right and wrong choices in 
a visual discrimination habit by rats 
produced more rapid learning. The 
animals spent more time at the 
choice point and apparently were 
more “deliberate” in their selection 
of pathways. 


Secondary Reward Training 


Token reward has been studied 
with rats by Skinner (1938), with 
cats by M. F. Smith (1939), with 
dogs by Ellson (1937), with chim- 
panzees by Wolfe (1936), Nissen 
and Crawford (1936), and Cowles 
(1937), and in a comparable situa- 
tion with children by Mitrano 
(1939). A secondary reward was 
used with rats by K. A. Williams 
(19292). The reward value of the 
sight of food has been demonstrated 
by Grindley (1929) with chickens, 
and by Bruce (1932) with rats in 
the maze problem. 


Chapter 4 
THE NATURE OF REINFORCEMENT 


HE STRENGTH of a conditioned response is increased by 

reinforcement and decreased by non-reinforcement. 
This is the basic statement of the conditioning process. Any 
theory about the conditioned response must concern itself 
with the concept of reinforcement; indeed, so central is its 
place in comprehending the functional relationships within a 
conditioning experiment that a theory may be judged by its 
success in handling this concept. 

A survey of the different kinds of conditioning experiments 
demonstrates clearly that the reinforcement is not described 
in the same way for the various situations. The attempt to 
formulate a single explanation of the reinforcement process 
which will encompass the diverse examples discussed in the 
previous two chapters has claimed the attention of many 
writers. Such theories have found their origin in an emphasis 
upon one or another of the fundamental reference experi- 
ments. The three outstanding principles are substitution, ef- 
fect, and expectancy. Substitution, the orthodox conditioning 
theory, is most appropriate as an explanation of learning by 
homogeneous reinforcement as it is studied in the classical 
conditioning procedure. The principle of effect applies most 
directly to learning by heterogeneous reinforcement in the in- 
strumental reward and escape situations. The expectancy 
principle draws its chief support from learning which demands 
the consideration of derived reinforcement as it is demon- 
strated in avoidance and secondary reward training. While 
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each principle appears to be coordinated with a particular 
variety of reinforcement, advocates of each principle have at- 
tempted to demonstrate that the principle can also explain 
learning which depends upon the other types of reinforce- 
ment. After a separate characterization of the principles, the 
claims for each as an exclusive theory of reinforcement will 
be considered. 


The Principle of Substitution 


The principle of substitution states that a conditioned 
stimulus, present at the time that an original stimulus evokes 
a response, will tend on subsequent presentations to evoke 
that response. This principle is fundamental in Pavlov’s the- 
oretical formulations, and, in one form or another, has been 
an essential part of most psychological treatments of the 
phenomena of association. The various statements of the prin- 
ciple of substitution differ principally in the designation of 
what is substituted. Thus early writers spoke of associations 
between ideas, or between sensations and impulses. Others 
have pointed out that associations must be conceived ulti- 
mately as relationships between neural processes in the brain. 
The principle is usually stated in terms of stimuli and re- 
sponses, however, since these are the events recorded in 
psychological experimentation. A more precise formulation 
would state that an activity initiated by a stimulus, occurring 
at the same time as another activity which results in a re- 
sponse, will tend on subsequent occurrences to evoke that 
response. 

The substitution principle has been applied to the phe- 
nomena of conditioning in two ways. The clearest example 
of this difference is found in the contrast between the theories 
of Pavlov and of Guthrie. Pavlov states that substitution will 
occur only under certain conditions, and will vary in degree 
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as those conditions approach optimal values. The tendency 
of a conditioned stimulus to evoke the response with which 
it has been paired depends upon the time interval separating 
the two, the intensity of the conditioned and unconditioned 
stimuli, the number of repetitions of the stimuli, and so forth. 

For Guthrie (1935), on the other hand, substitution occurs 
always and completely whenever a stimulus is present at the 
time that a response is made. Strict simultaneity is necessary, 
and when association apparently occurs between a stimulus 
and a response separated in time, the true conditioned stimulus 
must be identified as a stimulus present at the time of re- 
sponse. The external conditioned stimulus, such as a buzzer, 
is not necessarily the true conditioner; the true conditioned 
stimuli may be provided by the sensory consequences of in- 
tervening movements which the buzzer evokes. Learning is 
accomplished in every instance in one trial, and is permanent 
unless interfered with by new learning. One presentation of 
the conditioned stimulus on which the response does not occur 
suffices for unlearning. Stated more generally, this theory de- 
clares that the organism in a certain stimulus situation will 
inevitably do what it did on the last occurrence of that situa- 
tion. On the face of it, this theory seems to be in contradiction 
with the common observation that learning is slow and grad- 
ual and that responses show variability from one occasion to 
the next. The stimulus situation, however, must be thought 
of as including numerous component stimuli or cues, which 
are never exactly repeated in their entirety. Even if the en- 
vironmental objects are precisely the same, the organism will 
view them from a different position or angle. The cues arising 
from its own body will also be different. Responses will be 
identical on two successive trials only to the extent that the 
stimulus situations are identical. The reason that repetition 
of training trials is effective in increasing the probability of oc- 
currence or the magnitude of a conditioned response is that 
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the response becomes conditioned to the various changing 
forms of the stimulus situation which appear on successive 
trials. As more cues are assimilated to the response, adventi- 
tious stimuli will interfere less with the behavior, so that the 
response appears to be better learned.. 

These two conceptions of the principle of substitution sug- 
gest different experimental programs. Pavlov’s theory de- 
mands the empirical determination of the conditions affect- 
ing the process of substitution, and his followers have 
proceeded to study the influence of time interval, stimulus in- 
tensity, generalization, and so forth. Guthrie’s theory, on the 
other hand, does not require any study of such factors, for 
substitution always occurs completely in one trial. The prob- 
lem for Guthrie is essentially one of performance, of discover- 
ing the laws governing the occurrence of a response, since if 
the response occurs it is permanently conditioned to all stimu- 
lus cues present; if it does not occur it is permanently extin- 
guished. This experimental program would involve the identi- 
fication of the component cues in the stimulus situation and 
their separate response tendencies, and the statement of their 
mutual interrelations (facilitation and inhibition) in determin- 
ing the overt response. 

At the present time, Pavlov’s statement of the principle of 
substitution, while not so comprehensive as Guthrie’s, is more 
susceptible to experimental formulation. The difference de- 
pends principally upon the definition, or method of measure- 
ment, of the stimulus. Pavlov defines the stimulus in objec- 
tive terms independently of its effect upon the organism; the 
stimulus remains the same on successive conditioning trials 
and its changing relation to the conditioned response is 
studied. In Guthrie’s theory, a change in response from one 
occasion to another means a change in the stimulus situation. 
re ee not salivate te the bell on the second trial of a 

§ series, it is “because there are uncontrollable 
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differences in the situation on the second presentation” (Guth- 

rie, 1935, p. 100). But it is not possible to designate those 
stimulus changes; ordinary measurement of the stimulus does 
not reveal them. The only evidence of their presence is the 
very circular inference from the failure of the salivation re- 
sponse. Pavlov’s more empirical statement of the principle of 
substitution is directed toward the same objective as Guthrie’s 
but has the advantage that it can be checked and revised at 
each step of the way. 

The substitution principle, in any form, is characterized by 
the fact that no additional motivational principle is needed 
to account for learning. Motivation will obviously be a factor 
in determining the occurrence and magnitude of the original 
unconditioned response, but the substitution is dependent 
merely on the fact of response. Motivation may therefore be 
omitted from central consideration; it is one of a number of 
circumstances which influence the strength of the original re- 
sponse. 

One of the most difficult problems confronting any theory 
of conditioning is to account for the observed differences be- 
tween a conditioned response and the unconditioned response 
on which it is based. The principle of substitution, by itself, 
does not contain any explanation of these differences; indeed, 
it demands that the organism should act in the presence of 
the conditioned stimulus as though it were in the presence of 
the unconditioned stimulus. Something approximating this 
occasionally seems to be the case. Pavlov (1934) reports that 
his dogs, trained to salivate to a light preceding food, were 
noticed to turn their heads toward the light, and even to lick 
the light globe. Emotional conditioning, to the extent that it 
occurs, appears to be an attachment of the emotion aroused 
by one object to another that happened to be present at the 
time when the emotion was expressed. 

The most common result of classical conditioning, as pointed 
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out previously, is not complete but partial redintegration 
(Chapter 2, p. 37f.). The conditioned response resembles but 
does not duplicate exactly the unconditioned response. It may 
differ in magnitude and in form. Characteristically it is only 
one component of the original response. In many such in- 
stances it is apparent that the situation is also one of instru- 
mental training, and that component responses are strength- 
ened or weakened by heterogeneous or derived reinforcement. 
The principle of substitution cannot deal directly with re- 
sponses which show no resemblance to the original response, 
and in order to account for instances of partial identity, it is 
necessary to consider that some fractions of the response are 
more readily conditioned than are others. It is a common ob- 
servation that there are great differences in the speed of con- 
ditioning various responses (e.g., E. L. Walker and Kellogg, 
1939). 

It has been shown that the principle of substitution is di- 
rectly applicable to instances of complete redintegration 
which occur in the classical training situation, and to instances 
of partial redintegration which can be interpreted as the con- 
ditioning of a fraction of the original response. The possible 
extension of the principle to account for conditioning on the 
basis of heterogeneous and derived reinforcement will be con- 


sidered following the discussion of the principles of effect and 
expectancy. 


The Principle of Effect 


Studies of heterogeneous reinforcement in the typical in- 
strumental training situation have given rise to a principle of 
learning which emphasizes the influence of the consequences 
of an act, that is, the reward or punishment which follows 
the act. Although some such Principle may be found in the 


writings of earlier psychologists, it was first clearly stated by 
Thorndike in 191 1, who called it the law of effect: 
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Of several responses made to the same situation, those 
which are accompanied or closely followed by satisfaction to 
the animal will, other things being equal, be more firmly con- 
nected with the situation, so that, when it recurs, they will be 
more likely to recur; those which are accompanied or closely 
followed by discomfort to the animal, other things being 
equal, will kave their connections with that situation weak- 
ened, so that, when it recurs, they will be less likely to occur. 
The greater the satisfaction or discomfort, the greater the 
strengthening or weakening of the bond. (E. L. Thorndike, 
Animal Intelligence, The Macmillan Company, 1911, p. 244.) 


The discussions of the principle of effect have been con- 
cerned with two basic questions: (1) what kind of “effects” 
have a reinforcing action and (2) what responses will be re- 
inforced? The former question is essentially the problem of 
defining the nature of the stimuli which produce heterogeneous 
reinforcement, and will be considered first. 

Definition of Reinforcing Agents. Early statements of 
the effect principle were couched in the language of satisfac- 
tion and annoyance, related historically to the pleasure-pain 
theories embodied in various hedonisms. At the other extreme, 
the influence of satisfiers and annoyers on preceding behav- 
ior has been given a hypothetical neurological explanation in 
terms of changes in resistance to conduction along the ap- 
propriate nerve paths (Thorndike, 1911; Troland, 1928). 
Both the affective characteristics of rewarding and punishing 
agents and the neurological theory have assumed less domi- 
nant places in current statements of the effect principle 
(Thorndike, 1935). Experimental results have shown that 
acts are reinforced if followed by presentation of food when 
the animal is hungry, by water when the animal is thirsty, 
by moderate temperatures when the animal is hot or cold, 
by a sex object when the animal is in heat, by infant off- 
spring following parturition, by a free situation when the ani- 
mal has been confined, and by escape from electric shock and 
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other painful or uncomfortable situations. What do these 
situations have in common? Usually emphasized are either 
the consciously experienced pleasantness and unpleasant- 
ness, or the motivational aspects of the various reinforce- 
ments. 

The affective character of reinforcing stimuli received clear 
expression in Thorndike’s first statement of the law of effect. 
In other connections he has spoken of “satisfiers” and “an- 
noyers.” A similar conception is implied in Troland’s (1928) 
theory of retroflex action. Carr (1925) has emphasized the 
sensory aspect of the consequences of an act without specifi- 
cally stating that they must be affective. 

Many writers have attempted to restate the law of effect 
in such a way as to avoid the philosophical difficulties arising 
from the inference that affective states of consciousness have 
an influence on behavior, and to avoid the attribution of con- 
sciousness to animals. Thorndike, for example, states: “By a 
satisfying state of affairs is meant one which the animal does 
nothing to avoid, often doing such things as attain and pre- 
serve it. By a discomforting or annoying state of affairs is 
meant one which the animal commonly avoids and abandons” 
(1911, p. 245). Other definitions have emphasized the goal 
character of the reinforcing act. It has been pointed out that 
the animal in the problem-box or maze is aroused, restless, 
hypertensed, until the food is reached. After ingestion of food, 
the animal becomes calm and quiescent. S. Smith and Guthrie 
(1921) wrote of ‘maintaining stimuli? which are removed by 
the reinforcing act. In this they followed the conceptions of 
precurrent and consummatory response proposed earlier by 
Sherrington (1906) and Woodworth (1918). Raup (1925) 
described the end of goal activity to be the reaching of a state 
of complacency,’ similar to that which Cannon (1932) has 
called ‘homeostasis.’ T. L. McCulloch (1939c) has proposed 
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that reinforcement may be effective because it causes the dis- 
appearance of restless, excited behavior. 

Instrumental escape training has been distinguished from 
instrumental reward training in terms of the particular ar- 
rangements of experimental procedure which are customarily 
employed. Escape and reward, however, are similar in their 
dynamic action as reinforcement for learning, no matter what 
specific interpretation of the principle of effect is made. Termi- 
nation of an electric shock may be considered affectively 
pleasant or satisfying; it may be considered a consummatory 
response for the primary drive to avoid pain; or it may be 
thought of as a return to an equilibrium or state of com- 
placency. Indeed, reward might in some cases be considered 
an escape from the persisting stimuli which characterize the 
drive. Muenzinger and Fletcher (1936) have reported an ex- 
perimental comparison of hunger-food-tension and shock- 
escape-tension as reinforcement for the learning of a visual 
discrimination by rats. The effectiveness of each may be in- 
terpreted as an escape from tension present in the learning 
situation; in one case hunger, in the other, pain. Shaffer 
(1936) has also emphasized tension reduction as the essential 
characteristic of problem solution. 

It is apparent that no definition of effect provides an inde- 
pendent measure of the strength of a reinforcement. The 
degree of satisfaction, of complacency, or of tension reduction 
has not been objectively determined. The strength of rein- 
forcement can be given comprehensive definition only in terms 
of the amount of learning resulting from it. This is, of course, 
a circular definition, if strength of reinforcement is to be used 
as a factor determining degree of learning. A partial escape 
from circularity is achieved by the fact that a stimulus such 
as food which is found to be reinforcing in one situation will 
also be reinforcing in other situations, and with other animals. 
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At the present time it is possible to catalog the reinforcements 
which are almost universally effective, although it is not 
possible to formulate a single definition which will include 
all the examples. 

The original formulation of the law of effect stated that re- 
sponses which are followed by discomfort (i.e., punishment) 
will tend to be weakened in strength. This statement is ob- 
viously not a law of the acquisition of responses, but rather 
one of the reduction and elimination of responses. As such 
it is discussed more fully in the following chapter. As Muen- 
zinger and Dove (1937) have pointed out, punishment gives 
rise to variability of behavior, that is, to a tendency to do 
something else after failure, while reward leads to uniformity 
of behavior, or a tendency to repeat the response that was 
rewarded. 

Selection and Elimination by the Principle of Effect. 
The second question which has been prominently raised in 
discussions of the principle of effect concerns the relation of 
the reward to the response which it strengthens. An instru- 
mental training situation is ordinarily designed so that a cer- 
tain response will be followed by reward and thereby become 
selected, while other responses which do not lead to reward 
become eliminated in the course of training. The effective 
factor in determining the selection of the “correct” response 
is its proximity in time to the reinforcement. The last response 
which occurs prior to the reward is the one most strongly 
reinforced. Experimental results have shown that if the pres- 
entation of the reward is delayed for an interval of time, the 
speed of learning is correspondingly decreased. The relation- 
ship between the reward interval and the strength of the con- 
ditioned response may be described as the gradient of reward 

(see Chapter 7). 
> principle of effect does not require that the behavior 
quences strengthened by reward should necessarily be in- 
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strumental in securing the reward. The principle contains no 
implication of purposive behavior or of insight. The responses 
which are strengthened may be quite incidental to the achieve- 
ment of the reward, and under some circumstances may be 
responses occurring after the reward is given (p. 168). All that 
is necessary is proximity to the reinforcement. The principle 
of effect becomes a modified law of contiguity, differing from 
the principle of substitution in its statement of which events 
are temporally related. 

Comparison of the Principles of Effect and Substitu- 
tion. The principle of effect, like that of substitution, is de- 


Conditioned 
GIGHT = Lense 2s KNEE! JERK 


( Conditioned 
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FOOD 
(Reward) 








TENDON TAP 
(Unconditioned 
stimulus) 


Ficure 6 
COMPARISON OF THE PRINCIPLES OF SUBSTITUTION AND OF EFFECT 


According to the substitution principle the conditioned response depends 
upon the occurrence of the unconditioned knee jerk; according to the effect 
principle the conditioned leg extension depends upon the food reward. For 
further discussion, see text. 


signed to explain the occurrence and strengthening of the 
associative connection between a conditioned stimulus (situa- 
tion) and a certain response which is originally evoked by an- 
other stimulus. The difference between the two principles 
may be made clear by reference to a hypothetical training 
situation illustrated in Figure 6. A dog, for example, is pre- 
sented with a conditioned stimulus (light), followed by a 
stimulus which elicits a knee jerk, which in turn is followed 
by a reward (food). According to the principle of substitu- 


86 CONDITIONING AND LEARNING 


tion, the strength of the conditioned response (leg extension 
to light) depends upon the occurrence and the intensity of 
the unconditioned knee jerk response. The principle of effect 
states that the strength of the conditioned response is a func- 
tion of the occurrence and degree of food reward. Extinction 
is predicted by the substitution principle when the uncondi- 
tioned response (based on tendon tap) is omitted in a series 
of trials; the principle of effect demands that extinction follow 
the omission of the food reward. A further comparison is 
possible with reference to the spacing of the stimuli. The time 
interval between the conditioned stimulus and unconditioned 
response is crucial for conditioning according to the substitu- 
tion principle while the interval between the original re- 
sponse and the reward is important according to the effect 
principle. 

Direct comparisons of this sort between the principles of 
substitution and of effect are not frequently encountered in 
the experimental literature. In general, the substitution prin- 
ciple is applied in the explanation of the results of classical 
conditioning, while the effect principle is related to experi- 
ments involving instrumental training. Brogden (1939b) has 
recently studied a situation somewhat similar to the hypo- 
thetical example outlined above. Dogs were first trained, by 
pairing a bell and a shock stimulus, to give a conditioned leg 
withdrawal response, and then were rewarded with food when- 
ever this response was made. In spite of the omission of the 
unconditioned shock stimulus on as many as 1000 trials, 
there was no extinction of the conditioned withdrawal when 
it was followed by feeding. 

The effect principle, like the substitution principle, might 
conceivably be extended to cover instances of learning other 


than those from which it is primarily derived. This kind of 
extension of the term will be considered 


oe i after the expectancy 
principle has been discussed. 
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The Principle of Expectancy 


There are many instances of conditioning in which it is im- 
possible to identify the reinforcement in terms of an uncon- 
ditioned response or of an effect such as described above. 
Instrumental avoidance training, for example, cannot be ac- 
counted for directly by either substitution or effect. If re- 
sponse to shock is considered to be an unconditioned response, 
substitution would demand that when the shock stimulus is 
omitted on a series of trials, the conditioned response should 
become extinguished, whereas it actually is strengthened by the 
absence of shock. The principle of effect likewise provides no 
simple explanation, since the shock cannot be considered a 
reward, and the absence of shock can be rewarding only if 
some sort of expectation of shock has been established in the 
subject. 

The principle of expectancy has been formulated to ac- 
count for the strengthening of instrumental responses by de- 
rived reinforcement, as exemplified in avoidance and sec- 
ondary reward training. According to the principle of effect, 
reinforcement must be affectively pleasant or, stated other- 
wise, must consummate a primary drive. According to the 
principle of expectancy, reinforcement must be such as to 
confirm an expectancy. The expectancy itself depends upon 
previous learning by the subject in the particular situation or 
in similar situations. An expectancy may be thought of as a 
secondary or learned motive, differing from a drive in that 
it is characteristic of an individual and dependent on his prior 
experience, rather than being unlearned and universal for a 
species. The experiences of various individuals are so similar 
in many respects, however, that prediction of behavior on the 
basis of expectancies may be quite accurate. For example, in 
all animals the sight of familiar food reinforces instrumental 


approach or reaching responses. 
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Tolman’s theory of sign-learning is the most systematic 
formulation of the expectancy principle. According to Tol- 
man (1934), in learning a sequence of acts leading to a goal 
the subject follows the ‘signs’ which mark out the ‘behavior 
route’ leading to the ‘significate,’ or goal. The interpretation 
involves the perceptual or cognitive capacities of the animal. 
In the presence of the ‘signs’ the subject ‘expects’ the goal to 
appear if it follows the ‘behavior route.’ In other words, there 
are intermediaries (intervening variables) between the stimuli 
and the overt responses. The inference is made that the ani- 
mal behaves in certain ways consonant with anticipated con- 
sequences. If the goal materializes, the expectation is con- 
firmed, and when the situation is repeated similar behavior 
sequences will again be evoked. If the expected consequences 
do not materialize, failure will lead to varied behavior on 
the next trial. Such tentative behavior, entered upon in ac- 
cordance with expected consequences which are probable but 
not certain, will be immediately recognized by the reader as 
familiar in his own experience. Tolman’s theory of sign- 
learning, and Krechevsky’s (1932a) doctrine of hypotheses, 
interpret the behavior of animals in essentially these terms. 
These theories do not intend anything subjective, however, 
but they stress the organized relationships of the total be- 
havioral act, in which the goal is significant in relation to the 
expectations of it, whatever the structural basis of these ex- 
pectations may be. White (1936), following the lead of sev- 
eral earlier writers, has suggested that fractional anticipa- 
tory conditioned responses may provide the objective basis for 
expectations, reinforcement consisting in goal responses con- 
sonant with anticipatory responses. 

ji There are two steps in an explanation of the results of de- 
rived reinforcement according to the expectancy principle. 
The first step, encompassed by Tolman’s sign-learning theory, 
involves a learned sign-significate relationship, so that the 
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situation has a new meaning to the organism based on its 
prior experience in it. Thus the guinea pigs in the revolving 
cage of Brogden, Lipman, and Culler (1938) must learn that 
at the sound of the buzzer the cage is potentially ‘dangerous,’ 
i.e., a shock is likely to occur, and that the shock is less likely 
to occur if they start running when the buzzer sounds. Wolfe’s 
(1936) chimpanzees must know that tokens signify potential 
food. The second step of explanation requires some manner 
in which the aroused expectation or signification leads to ap- 
propriate conduct. It is assumed by Tolman that if a rat 
knows the behavior-route to a desired goal, the rat will follow 
the route. He has been criticized by Guthrie (1935, p. 172) 
for not coming to grips with the problem of how the rat is 
led from its expectations to appropriate action. Both the sub- 
stitution and effect theories refer to strengthened tendencies 
to respond; what is strengthened according to the expectancy 
theory is not a tendency to respond, but a tendency to antici- 
pate certain consequences. The theory, stated in this form, 
does not propose to predict the details of conduct, so that 
the most varied behavior, if it is consonant with anticipated 
consequences, may be encompassed by it. A rat expecting 
punishment at the end of the maze may refuse to enter, may 
climb over the wall, or may simply crouch in the path and 
shudder. It shows in its behavior that it anticipates a noxious 
consequence of running through the maze, but the details of 
the behavior are not accounted for by the mere acquisition of 
an expectation. Widely different performances may serve the 
same purposes, and hence be grouped together in, say, the 
class of responses going by the name of avoidance behavior. 
This breadth of interpretation is one of the advantages of the 
expectancy principle over the principles of substitution and 
effect, but it is at the same time a source of weakness. All 
that is learned, according to the expectancy principle, is a 
new anticipation. The appropriate behavior in relation to the 
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anticipation must have been either inherited or acquired in 
the earlier history of the organism. Zener (1937) has used the 
expression ‘sign-urge’ to refer to his form of the expectancy 
theory in which he recognizes that signs must be related to 
specific tension systems in order for the behavior to be ex- 
plained. An advance will be made in explanation according to 
the expectancy principle when more detailed analyses are 
given of the way in which signification leads to conduct. 


Generality of the Principles 
of Reinforcement 


Each of the three principles of reinforcement has been dis- 
cussed with particular reference to the type of training ex- 
periment from which it was originally derived. Proponents of 
each principle, however, claim that it will account for all the 
phenomena of learning. These claims may now be examined 
in order to determine the success with which any single prin- 
ciple can encompass the results of conditioning in general. 

Principle of Substitution. Although Pavlov’s writings 
imply that substitution is the only principle necessary for the 
explanation of all varieties of learning, they contain no specific 
discussion of the problems arising from the consideration of 
instrumental reward or avoidance training. Other writers, 
however, have dealt directly with this question. They do not 
deny that learning takes place in situations involving hetero- 
geneous or derived reinforcement, but believe that such learn- 
ing can be explained without any new principles beyond that 
of substitution. 

The explanation of instrumental reward training, such as 
is involved in a problem-box task, requires a statement of why 
the correct response, and not some other, becomes strength- 
ened so that on subsequent trials it is more readily elicited 
by the conditioned stimulus. Attempted explanations in terms 
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of secondary principles of frequency, recency, and intensity 
have met with little success. Guthrie’s more recent statement 
of the principle of substitution permits him to account for 
such learning without any additional assumptions. The ani- 
mal, in a problem situation, always tends to do what it did 
on the last presentation of that situation. Since the correct re- 
sponse is invariably the last response in the situation, it is 
the one which is learned. Incorrect responses are not perma- 
nently learned because the animal remains in the presence of 
the conditioned stimulus, and as new responses occur, the 
previous ones are unlearned. Since learning and unlearning 
require only a single occurrence, it is apparent that the last 
response made to the conditioned stimulus is the only one 
which is associated. When next the animal is placed in the 
situation, it does what it last did; what it last did is always 
the response just preceding success. 

This explanation differs clearly from the principle of effect, 
according to which the reward is described in terms of af- 
fective pleasantness, consummation of drive, or tension re- 
duction. In Guthrie’s theory, while the statement of effect 
remains descriptively appropriate, the important function of 
the reward is merely to terminate the particular complex of 
conditioned stimuli. This is usually accomplished by removal 
of the external environmental stimuli. In the experiments of 
Guthrie and Horton (1937) in which cats escaped from a 
problem box to a food reward, the learning appeared to de- 
pend very little on the consequences of eating, since the cats 
often refused the food after successfully operating the mechan- 
ism for getting out of the box. While escape may itself have 
been tension-reducing, the important point for the theory 1s 
that leaving the box constituted a terminal act in relation to 
the stimuli present while the animal was in the box. In situa- 
tions in which the animal is rewarded in the same environment, 
there is still a marked change in the total stimulus complex, 
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for the hunger stimuli are presumably terminated by the re- 
ward. According to this interpretation, it is the capacity to take 
the animal out of the situation, to terminate the conditioned 
stimulus, which makes heterogeneous reinforcement effective 
in learning. 

Guthrie’s explanation of instrumental learning has the same 
advantages and disadvantages as his general theory of sub- 
stitution. In spite of its consistent logical structure, it does 
not lend itself readily to precise experimental verification. 
The action of reward in terminating the stimulus situation 
cannot be clearly specified in many instances. For example, 
when a rat eats a pellet of food in the Skinner problem box 
the stimulus situation is not materially altered. The animal 
remains in the same box, and the drive stimuli are not elimi- 
nated since the animal will continue to eat many more pellets. 

In response to criticism of this sort by T. L. McCulloch 
(1939c), Guthrie (1939) has recently proposed that the act of 
eating removes the conflicting behavior which is the ‘annoy- 
ance, whether or not it removes the internal state which is the 
‘annoyer.’ Hence there is tension reduction in the Skinner ex- 
periment as the rat eats a pellet, even though it remains ~ 
hungry enough to eat many more. The temporary tension 
reduction is a terminal act removing stimuli, and hence pre- 
vents unlearning of responses based on these stimuli. If 
Guthrie’s argument holds, however, as soon as the pellet is 
consumed and the organism is again in an aroused state it is 
in a situation favorable for unlearning all that it learned un- 
less it immediately repeats successfully its food-seeking be- 
havior. It is obvious that something has changed when the 
animal ceases to run or to push levers and begins to eat, but 
E theory must state more precisely just what the change is 
which guarantees the learning of the prior response if the 
theory is to be verified experimentally. 


Explanation of the effects of derived reinforcement in terms 
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of the principle of substitution has been undertaken by Hull 
(1931) and by Guthrie (1935). Both assign major importance 
to fractional anticipatory conditioned responses as surrogates 
for expectancy. The complexity of the problem, and the im- 
plicit character of many of the relevant events, make this 
explanation involved and difficult to verify. A more extended 
discussion of this question will be presented in Chapter Io. 

Principle of Effect. Although it was derived from experi- 
ments involving heterogeneous reinforcement, the principle of 
effect might be extended to account for instances of homo- 
geneous and derived reinforcement and thereby give rise to 
a uniform and comprehensive theory. In order to include the 
results of classical conditioning under the principle of effect, 
it would be necessary to demonstrate that the conditioned re- 
sponse is always followed by reward or escape. In the case of 
Pavlov’s experiments, for example, the conditioned salivation 
is always followed by the presentation of food. In the case of 
withdrawal from shock, the shock may be considered as a 
punishment as well as an unconditioned stimulus. Even 
though the conditioned response does not prevent the shock, 
the conditioned movements may be interpreted as reactions 
related to punishment. 

The most serious difficulties in explaining classical condi- 
tioning by the principle of effect are encountered in experi- 
ments in which the unconditioned stimulus does not seem to 
have any of the characteristics of a reward either intrinsically 
or in a derivative sense, i.e., it is not satisfying or annoying, 
it is not tension reducing, and it does not consummate a drive. 
Close approximation to this is found in the conditioned lid 
response based on sound stimulation, and in the conditioned 
pupillary constriction based on light stimulation. The prin- 
ciple of effect also seems inapplicable to cases of ‘incidental 
learning,’ in which mere repeated exposure appears to pro- 
duce learning without reference to any reinforcing state of af- 
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fairs. A clear example of such incidental learning is furnished 
by an experiment by Brogden (1939c). A light and a sound 
were presented together to a dog for a number of trials, after 
which a conditioned foot retraction was established to the 
sound stimulus. Tests demonstrated that the light would also 
evoke the conditioned response, although it would not in other 
dogs which had not previously received the light-sound train- 
ing. Similar difficulty is created by the facts of latent learning 
(Blodgett, 1929). A rat in a maze without food reward does 
not exhibit error-reduction in successive trials comparable 
to a rat fed each trial at the end of the maze. So far the effect 
principle appears to be appropriate. But it may be demon- 
strated that learning was taking place in the absence of food- 
reinforcement, by introducing food in the later trials of the 
series. Now the rats not previously fed in the maze very 
promptly perform as satisfactorily as the control animals 
which were fed each trial. The change from the poor non- 
rewarded performance to the good food-rewarded perform- 
ance occurs so rapidly as to reflect earlier learning not readily 
explained in terms of effect. 

Derived reinforcement can be subsumed under the prin- 
ciple of effect only by redefining reward to include secondary 
rewards. A step toward unification of theory is taken, how- 
ever, if it can be shown that derived reinforcement is based 
upon learning which involves the effect principle. It may 
often be demonstrated directly that the situation producing 
derived reinforcement loses its value unless it is supported, 
directly or indirectly, by heterogeneous reinforcement. Culler, 
Finch, Girden, and Brogden (1935), when using the buzzer 
as a substitute for shock, found it necessary to introduce the 
shock occasionally as a “reminder.” The sight of food loses 
its secondary reward character if it is not itself reinforced by 
actual feeding (Grindley, 1929). A sign appears to derive 
its effectiveness as reinforcement from the degree to which 
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it substitutes for an agent capable of effect reinforcement. 

The Principle of Expectancy. To the extent that condi- 
tioned responses may be interpreted as readiness or prepara- 
tory acts, or as behavior correlated with sets or attitudes 
anticipatory of reinforcement, they fall under the general con- 
cept of expectancy, whether or not reinforcement appears 
descriptively to be homogeneous, heterogeneous, or derived. 
In order to extend the expectancy explanation, classical con- 
ditioning must be subordinated to goal-behavior. This is 
similar to the subordination of substitution to effect, except 
for the interpretation. The acts which are strengthened must 
be interpreted as cognized behavior-routes leading to the sat- 
isfaction of expectancies. Redintegration which is unrelated 
to goals cannot be explained according to the expectancy prin- 
ciple. To retain the principle of expectancy as an exclusive 
explanation of learning, simple substitutive or redintegrative 
learning must be denied. 

To the extent that reward reinforcement depends upon a 
learned anticipation of the reward, the expectancy principle 
is more appropriate than effect. It has been demonstrated in 
certain experimental situations that what appears to be law 
of effect reinforcement may involve the expectancy of a spe- 
cific goal object. An illustration is provided by Tinklepaugh’s 
(1928) experiments in which a monkey rejected a lettuce- 
leaf when expecting a banana as reward, although lettuce was 
acceptable under other circumstances. The animal was work- 
ing for a reward, but its behavior is better described in terms 
of expectancy than in terms of effect. That chimpanzees act 
in accordance with specific reward-expectancy has been dem- 
onstrated by Cowles and Nissen (1937). 

An essential supplement to the principle of effect is the 
statement of the gradient of reward; i.e., that a response is 
strengthened in relation to its proximity to the reward. Ac- 
cording to the principle of expectancy, this relationship might 
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be reinterpreted as a gradient of uncertainty. Thus, in a 
linear maze, animals tend both to anticipate and to overshoot 
a turn leading to food (Spragg, 1936). This may be explained 
as lack of certainty with respect to the exact location of the 
turn. The entrances chosen tend to be near the correct one, 
with a decreasing number of entrances farther from the cor- 
rect turn. In the multiple choice word-number experiments of 
Thorndike (see p. 167f.), when one response is called “Right,” 
the effect may spread to neighboring sequences because of 
two kinds of uncertainty: an uncertainty as to the exact pair 
which was rewarded (corresponding to the uncertainty with 
respect to the correct turn in the maze), and an uncertainty 
as to why this was rewarded. To play safe, the subject will 
tend to repeat pairs near the successful one to the extent that 
such repetition is possible. The gradient of reward would 
thus be accounted for as a spread of uncertainty. Some doubt, 
however, is cast on this interpretation by the experiments of 
Muenzinger and his associates (Muenzinger and Dove, 1937, 
and Muenzinger, Dove, and Bernstone, 1937). In the first of 
these, the Thorndike word-number experiment was repeated. 
To each of 20 words a number from 1 to 10 was assigned by 
the experimenter. As each word was read to the subject, he 
was to reply with a number from 1 to 10, and the experimenter 
announced “Right” or “Wrong” following each reply. In 
harmony with Thorndike’s finding, it was found on repeti- 
tion that there was a tendency to repeat responses made near 
to those called “Right,” even though these responses had been 
‘punished’ by the previous announcement of “Wrong.” In a 
similar experiment, uncertainty was reduced by teaching in 
advance those word-number sequences which would be pro- 
nounced “Right” by the experimenter. In spite of this prior 
training there was a spread of reinforcing effect to items just 
r e and following the item reinforced by saying 

Right.” In a second experiment, the spread of reinforcement 
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was determined for rats in an endless maze. Here responses 
both prior to and following food reward were found to be 
strengthened. The gradient of reward is therefore a double 
one, affecting responses beyond the reward as well as those 
before it. There is no easy way for an expectancy theory to 
account on an uncertainty principle for the strengthening of 
behavior following eating. It is not plausible that the rat was 
so confused as to believe that its prior eating depended on 
its subsequent alley behavior. 

The expectancy principle has a distinct advantage over the 
others in cases in which reinforcement does not fulfill the re- 
quirements either for substitution or for effect. If the present 
limitations of vagueness and lack of quantitative verification 
are Overcome, expectancy may become a very important aid 
in the interpretation of experimental results in learning. 


Present Status of the Problem 
of Reinforcement 


The foregoing arguments have illustrated the attempts to 
include the results of all three varieties of reinforcement under 
each of the three explanatory principles. Implicit in the dis- 
cussion are several distinctions which have not always been 
recognized in the various attempts to summarize and compare 
theories appropriate to different experimental situations. The 
empirical starting point is results achieved within experimental 
arrangements which throw into focus particular behavioral 
relationships requiring explanation. We have considered three 
main groups of experiments: (1) classical conditioning, (2) 
reward and escape training, and (3) avoidance training and 
secondary reward training. It is a common error to permit the 
reference experiment to dramatize a particular process, and 
then to suppose that the experiment represents a pure case of 
the process dramatized. Thus the classical conditioning experi- 
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ment of Pavlov throws into relief the attachment of the con- 
ditioned stimulus to the unconditioned response, but it is by 
no means a pure case of one stimulus substituting for another. 
Similarly, an experiment involving rewards is not a pure case 
of learning according to the law of effect. Having chosen cer- 
tain reference experiments which by the nature of the case 
could not be pure illustrations of one or another process in iso- 
lation, we have inferred particular processes prominent in each 
of the reference experiments. Through abstracting from the 
total experiment we have, in a sense, artificially created the 
pure case. Thus from classical conditioning we have abstracted 
the process of homogeneous reinforcement, in which behavior 
is strengthened through being followed by similar behavior; 
from reward and escape training we have abstracted the proc- 
ess of heterogeneous reinforcement, since the learned behavior 
and rewarding behavior differ markedly; and from avoidance 
learning and secondary reward we have abstracted the process 
of derived reinforcement, since reinforcement is effective in 
the absence of the stimulus giving rise to the original motiva- 
tional reinforcement yet is dependent upon this original re- 
inforcement. These abstracted processes are purified aspects 
of experimental situations, each process suggested by an ap- 
propriate reference experiment, but not to be identified with 
it. Having abstracted these processes from the experiments, 
we found certain types of explanations closely coordinated 
with each process. Thus the principle of substitution appears 
to be an appropriate explanation of homogeneous reinforce- 
ment; effect is appropriate to heterogeneous reinforcement; 
expectancy is appropriate in explanation of derived reinforce- 
ment. But those who favor one or another of these explana- 
tions do not rest content with it as an explanation of that va- 
riety of experiment to which it is most appropriate, but seek 
to interpret the processes within the other experiments as well. 
The accompanying table summarizes these attempts. The ef- 
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forts to achieve a unified explanation of all varieties of rein- 
forcement according to one or another of the explanatory 
principles cannot be considered final. 


TABLE 2 


How Eacu Princrrre May Be Proposep as THE EXCLUSIVE EXPLANATION OF 


THE LEARNING Process UNDER CONSIDERATION 


Type of Abstracted 

experiment process 

1. Classical Homogeneous 
condition- reinforcement 
ing 

2. Instru- Heterogene- 
mental re- ous reinforce- 
wardand ment 
escape 

3. Instrumen- Derived re- 
tal avoid- inforcement 
ance; sec- 
ondary 
reward 


Principle of explanation as applicable 


SUBSTITUTION 
Substitution prin- 
ciple is directly 
applicable 


Reward (or es- 
cape) terminates 
conditioned stim- 
ulus and the last 
response madere- 
mains conditioned 


Conditioned 
stimulus evokes 
fractional antici- 
patory responses 
learned by sub- 
stitution. These 
are the surrogates 
of expectancy 


to each process 
EFFECT 
Conditioning 
depends upon 
drive and het- 
erogeneous re- 
inforcement 


Effect principle 
is directly ap- 
plicable 


Heterogeneous 
reinforcement 
is necessary to 
produce and 
support derived 
reinforcement 


EXPECTANCY 
Learning occurs 
only if response 
is part of a be- 
havior-route to a 
goal 


Reward (or es- 
cape) confirms 
expectancy; what 
appears to be a 
spread of effect is 
a gradient of un- 
certainty regard- 
ing the probabil- 
ity that expected 
consequences will 
materialize 


Expectancy prin- 
ciple is directly 
applicable 


It should be clearly understood that there is no logical neces- 
sity for reducing all instances of learning to a single explana- 
tory principle. The alternative theoretical systematization 
permits the retention of several principles of learning by 
clearly defining the situation to which each is applicable. Most 
of the writers who have attempted to deal comprehensively 
with the problem of learning have adopted the latter point of 
view, speaking of ‘types’ or ‘levels’ of learning. In these cases, 
however, the scope of application of each principle has not 
been given a precise and independent definition. It has been 
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pointed out above that differences in experimental training 
procedures do not provide a basis for distinguishing the vari- 
ous types of reinforcement. In the classical conditioning ex- 
periment, for example, homogeneous, heterogeneous and de- 
rived reinforcement may all be operative. A more useful 
proposal has been advanced by Skinner (1938). According 
to this theory, Type S conditioning (corresponding to homo- 
geneous reinforcement) occurs in the case of respondent 
behavior, while Type R (corresponding to heterogeneous re- 
inforcement) occurs in the case of operant behavior. Unfor- 
tunately the distinction between respondent and operant is 
not always clear. A particular response, such as the eye-blink, 
may under different circumstances be ‘elicited’ as a respond- 
ent or ‘emitted’ as an operant, and it becomes difficult to 
decide to which category a conditioned eye-blink belongs. 
Skinner (1938, p. 112) has tentatively suggested that Type S 
conditioning might be limited to the smooth muscle and 
glandular responses under the control of the autonomic nerv- 
ous system, while Type R conditioning would apply to skeletal 
muscle responses under the control of the somatic nervous 
system. This distinction could be supported only if further 
analysis demonstrated that the conditioning of responses 


such as leg flexion, eyelid closing, and the knee jerk did not 
involve the principle of substitution. 


Notes 


The various theories of learning 
discussed in this chapter have devel- 
oped quite independently of each 
other, and have only recently been 
coordinated in such a way as to per- 
mit their direct comparison. Condi- 
tioning experiments and Pavlovian 
theory became almost indissoluble 
through thirty years of close identi- 
fication. The notion that the condi- 


tioned response was simply the origi- 
nal unconditioned response attached 
to a new stimulus was conceptually 
so simple, and fitted so nicely the 
habit of thought inherited from as- 
sociation psychology, that it long re- 
mained unquestioned, and even to- 
day is often adhered to by those 
little acquainted with the details of 
conditioned response experiments. 
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The Principle of Substitution 


The classical and orthodox condi- 
tioning theory is derived from the 
law of contiguity in association psy- 
chology. Although Warren’s state- 
ment that the primary laws of asso- 
ciation may be reduced to the single 
law of contiguity (H. C. Warren, 
1921, p. 289) has not been accepta- 
ble to all associationists (Robinson, 
1932, p. 66), the increasing promi- 
nence of contiguity represented a 
trend within association theory to 
which simultaneous conditioning was 
readily assimilated. The change came 
chiefly in what items were said 
to be associated. To ideas, various 
associationists had added sensory 
processes, complete perceptual acts, 
affective activities, sets, muscular 
movements, glandular secretions, in- 
hibitions (Robinson, 1932, p. 27ff.). 
The emphasis on stimuli and move- 
ments represents an attempted 
objectification and simplification. 
Guthrie has been taken as represent- 
ative of other proponents of a sub- 
stitution theory, important among 
whom may be mentioned Holling- 
worth (1928), Holt (1931), Meyer 
(1934). 

The differences between the points 
of view of Pavlov and Guthrie are 
brought out in a controversy be- 
tween them, Pavlov (1932b), Guth- 
rie (1934). 


The Principle of Effect 


Thorndike’s law of effect, an- 
nounced in his Animal Intelligence 
(1911), has been the source of much 
controversy. For summaries, see 
Stephens (1931a), Cason (1932), 
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Waters (1934), Carr and others 
(1938). Cason has reviewed the 
early statements of the principle by 
Spencer, Bain and Baldwin. 

The most extensive experiments 
giving empirical comparison of sub- 
stitution and effect are those of Ko- 
norski and S. Miller (1928-1937). 
Unfortunately, few of the details are 
available in the more familiar lan- 
guages. Other experiments bearing 
on the problem are those of Anokhin 
and Straj (1937), Brogden (1939b), 
W. T. James (1933, 1934a), Sinke- 
vich (1929). 


The Principle of Expectancy 


The striving, goal-seeking charac- 
ter of learned behavior, while to 
some extent recognized in condition- 
ing and trial-and-error theories, has 
been most emphasized in purposive 
or hormic theories, such as that of 
McDougall (1923). Field or equi- 
librium theories may emphasize the 
directed character of behavior with- 
out subscribing fully to the avowed 
teleology of McDougals purpo- 
sivism (Adams, 1931; Humphrey, 
1933; Tolman, 1932). Recognition 
of the tendency for a learned se- 
quence to progress toward a sys- 
tematically related end-result was 
expressed in the theories of confirma- 
tion (Hobhouse, 1901), congruency 
(S. J. Holmes, 1911), and complete- 
ness of response (Peterson, 1916). 
White’s (1936) completion theory 
of reinforcement takes account of 
the advances in interpretation of 
conditioning phenomena. 

The expectancy principle, involv- 
ing as it does perception of relation- 
ships, cognitive functions, anticipa- 
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tions of consequences, hypotheses, 
appears to be quite foreign to con- 
ditioning theory. Closer scrutiny will 
show, however, that doctrines of 
symbolization, set, attitude, expecta- 
tion, are not uncommon in the liter- 
ature of conditioning. Pavlov himself 
is responsible for calling the condi- 
tioned stimulus a signal; Gantt, his 
student and translator, has coordi- 
nated Pavlov’s signalization with 
Adolf Meyer’s symbolization (Wolff 
and Gantt, 1935). Liddell, James, 
and Anderson give an expectancy in- 
terpretation of conditioning in sheep 
when they state that “when warned 
of an approaching electric shock the 
‘conditioned’ animal seeks to defend 
itself in the manner appropriate to 
the situation” (1934, p. 54f.). The 
clearest acceptance of an expectancy 
interpretation of the Pavlov experi- 
ment is by Zener (1937), whose ob- 
servations were quoted earlier (p. 
39). Attitude and expectancy have 
been found to be of paramount im- 
portance in human conditioning 
(Chapter 11). 


Generality of the Principles of 
Reinforcement 


Guthrie’s explanation of instru- 
mental reward learning in terms of 
substitution involves no additional 
assumptions, since the acquisition of 
the last act performed in a situation 
follows as a corollary of his concep- 
tion of one trial learning. Other writ- 
ers who reject the principle of effect 
have used supplementary principles 
to account for the strengthening of 
the correct response in rewarded 
learning. Among these principles are 
frequency of occurrence and recency 
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in time (Watson, 1914), and in- 
tensity of response (Holt, 1931). 
The frequency-recency theory of 
Watson did not survive the many 
criticisms made of it (e.g., Peterson, 
1922; Gengerelli, 1928). Holt’s in- 
tensity theory is that the correct re- 
sponse is retained because it is pro- 
longed and facilitated by a burst of 
proprioceptive impulses from mus- 
cles involved in the terminal act. 
Observation of animals does not of- 
fer any confirming evidence; the 
successful response is not necessarily 
prolonged but is usually quickly in- 
terrupted by movements of ap- 
proach to food and eating. 

Thorndike does not insist upon 
effect as an exclusive principle. He 
assigns a subordinate rôle to what 
he calls associative shifting (1935, 
p. 191), and includes an admixture 
of expectancy in some of his more 
recent statements about effect, such 
as that the effect is a confirmatory 
reaction or, in some cases, knowledge 
of results is validating (1935, p- 28)- 
Hull, while distinctly excluding a 
separate expectancy principle, has 
implied stimulus-substitution in the 
form of redintegration particularly 
in his earlier papers. His fractional 
anticipatory responses and his pure- 
stimulus acts, were derived, not in 
accordance with the principle of ef- 
fect, but in accordance with a sub- 
stitution principle. More recently, 
however, the principle of effect has 
become for him an exclusive princi- 
ple (Hull, 1937). 

Tolman similarly does not believe 
expectancy (i.e. sign-learning) tO 
be an exclusive explanation of learn- 
ing. His point of view is rather that 
some such theory is necessary to €x- 
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plain certain types of learning (Tol- 
man, 1937). Other kinds of learn- 
ing may be explained in other ways. 
In the paper referred to, Tolman 
lists seven sub-varieties of learning, 
each of which may have its own 
laws. Substitute-stimulus learning 
and trial-and-error learning are in- 
cluded among these. In fairness to 
Tolman’s position, it must be point- 
ed out that the relation of sign 
learning to the other varieties (sub- 
stitute-stimulus, trial-and-error and 
so on) is not a mere additive one. 
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For him the associationist account 
cannot be made complete by includ- 
ing a further variety of reinforce- 
ment. Tolman believes a radical de- 
parture from association theory to 
be necessary, and objects to the in- 
clusion of his suggestions within a 
framework of association theory. 
Hence our inclusion within condi- 
tioning of results based on derived 
reinforcement explained on the basis 
of expectancy is not a reflection of 
Tolman’s own position, although it 
owes much to him. 


Chapter 5 
THE NATURE OF EXTINCTION 


EINFORCEMENT has been shown to be the essential condi- 
R tion for the strengthening of a conditioned response. 
Omission of reinforcement leads to the weakening of a con- 
ditioned response, as illustrated in the experimental procedure 
of extinction. If the conditioned stimulus is repeatedly pre- 
sented without its usual reinforcement, the conditioned re- 
sponse undergoes a progressive decrement and finally fails to 
occur. A dog, for example, which has learned to run to the 
kitchen for food whenever it hears a pan rattled on the floor 
will gradually cease to respond if it no longer finds food in the 
pan. In the laboratory, a dog will gradually blink less fre- 
quently and less vigorously to a conditioned stimulus which 
is no longer followed by the customary puff of air to its eye. 

This reduction is not a spontaneous decay. Conditioned re- 
sponses show a very limited tendency to diminish with the 
passage of time. In the absence of complicating factors there 
is very little reduction by disuse even over intervals of months. 
It is apparent, therefore, that the elimination of conditioned 
responses depends upon some active process. This is called 
inhibition, which may be defined as the reduction in strength 
of response resulting from positive stimulation of some sort. 
To call this inhibition explains nothing, but serves to differ- 
entiate this kind of reduction from the decrements produced 
by changes in the conditioned stimulus itself, such as decreased 
intensity or qualitative alteration, by diminution in strength 
of drive, by administration of drugs, and so forth. The use 
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of the term inhibition implies that the reduction in strength 
of response satisfies three conditions: (1) the inhibited re- 
sponse is not permanently destroyed or abolished, but rather 
is suppressed or prevented from overt expression; (2) the 
stimulus for the inhibited response is the same stimulus which 
previously evoked the response with greater strength; and 
(3) the inhibition depends upon positive stimulation. 

Inhibition of conditioned responses results from several dif- 
ferent kinds of experimental procedure, such as extinction, 
discrimination, delay, distraction and so forth (Chapter 2). 
These various procedures appear to involve two basic proc- 
esses. The first, intrinsic inhibition or adaptation, may be de- 
fined ae the reduction in response resulting from continued 
elicitation of the response itself. The second, extrinsic inhibt- 
tion or interference, may be defined as the reduction resulting 
from the simultaneous elicitation of another, incompatible re- 
sponse. In many of the situations in which inhibition is ob- 
served, either or both types are possible. Before considering 
such situations, of which extinction is the most prominent ex- 
ample, we will examine the characteristics of the two kinds of 
inhibition separately. 


Adaptation 


The essential condition for adaptation is the repeated elici- 
tation of a response. This effect is not limited to conditioned 
responses but can be observed in any type of reaction. 

Adaptation of reflexes is commonly termed habituation or 
negative adaptation. It has been studied in many different 
organisms, ranging from the simplest to the most complex. It 
has been demonstrated in spinal cats (Sherrington, 1906) and 
rats (Prosser and Hunter, 1936). The degree of decrement 
varies widely from one response to another. Startle reflexes 
are rapidly habituated; flexion reflexes show more decrement 
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than tendon reflexes, while some responses, like the pupillary 
reflex or the respiratory reflexes, are apparently completely 
resistant to adaptation. 

Adaptation in voluntary responses which are made to in- 
structions is known as work decrement. It shows many of the 
same characteristics as reflex habituation (Robinson, 1934). 
Adaptation will of course be opposed by any factor tending to 
strengthen or maintain the response during repetition. Such 
factors have not been identified for reflexes, but instructions 
and incentives will prevent decrement in voluntary responses, 
while reinforcement will overcome adaptation effects in the 
case of conditioned responses. 

The chief characteristics of adaptation can be summarized 
best from the results of studies which have dealt either with 
the habituation of reflexes or with the extinction of conditioned 
responses, in which the factors opposing adaptation are at a 
minimum. Among these characteristics are the following: 

1. Adaptation is cumulative. The greater the number of 
repetitions, the greater the decrement. Although the course 
of adaptation is progressive, the rate is not constant, and 
curves plotting strength against repetitions show many minor 
variations and irregularities. After the overt response has com- 
pletely disappeared, adaptation may continue “below zero” 
as shown by the longer time required for spontaneous recov- 
ery. This may also be demonstrated directly by using a more 
sensitive measure of response. By recording very slight incip- 
lent movements with a sensitive tambour (Brogden, Lipman 
and Culler, 1938), or by recording action potentials from the 
muscles involved (Hilden, 1937), it has been shown that ex- 
tinction continues after all grossly observable movement has 
disappeared. 

2. Adaptation is a function of the rate of elicitation of the 
response. Although the evidence on this point is far from con- 
clusive, it appears that a faster rate of stimulation results in 
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more rapid adaptation, whether the responses studied are re- 
flexes, conditioned responses, or voluntary responses. 

3. Adaptation 1s not permanent. During an interval of time 
following adaptation, the inhibited response spontaneously re- 
covers in strength. Recovery begins immediately so that the 
interpolation of a short rest interval during adaptation will 
produce some increase in response strength. The rate and ex- 
tent of recovery vary greatly for different responses. A second 
adaptation following recovery from the first usually requires 
fewer repetitions for complete inhibition of the response. This 
is evidence that spontaneous recovery tends to be incomplete. 
To the extent that the effects of adaptation are temporary, it 
may be inferred that the response tendency is suppressed and 
not destroyed, although the response itself may be reduced 
to zero magnitude. 

4. Adaptation may be generalized to some degree. Experi- 
mental results on the extinction of conditioned responses show 
that the adaptation is not restricted to the particular response 
being elicited, but is reflected in the weakening of other re- 
sponses as well. This is called secondary extinction by Pavlov, 
who pointed out that the decrement may in some instances be 
quite widespread, leading to sleep-like states. 

Generalization of adaptation has not been widely studied 
with reflex habituation. Porter (1938a) noted that adaptation 
of the galvanic skin reflex to one stimulus resulted in a de- 
crease in the response to another stimulus, from one modality 
to another, as well as within a modality (Coombs, 1938). 
However, in adaptation situations with spinal animals (Sher- 
rington, 1906; Prosser and Hunter, 1936), no generalization 
is observed. A slight change in the point of application of the 
stimulus to the skin is followed by a response of full strength. 
This may mean that generalization is not an essential feature 
of the adaptation process itself, but depends upon the func- 
tioning of the higher brain centers. 
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5. The stimulus to an adapted response may be inhibitory, 
rather than neutral. Although the stimulus no longer elicits its 
original response, it may yet have an effect on other responses 
of the organism. Pavlov has demonstrated this clearly by 
showing that such an inhibitory stimulus, presented together 
with an ordinarily effective conditioned stimulus, will inhibit 
the response to the latter (1927). This result is similar to the 
phenomenon of conditioned inhibition. No data are available 
to indicate whether or not the same is true of the stimulus for 
a reflex which has been habituated. 


Interference 


Adaptation refers to a decrement for which the sufficient 
condition is the repetition of the same response. Interference, 
as an explanation of decrement, involves the interaction of 
two or more responses. 

The general principles of the interaction of simultaneous 
responses have been most carefully worked out in the field of 
reflex activity. Presentation of two stimuli which individually 
elicit identical or synergic responses, will result in summation 
or facilitation, the combined response being larger than that 
to either of the two separate stimuli, often larger than their 
sum. If the two reactions are antagonistic, however, interfer- 
ence results. ‘This may take the form of a complete suppres- 
sion of one response, partial suppression, or alternation of 
the two responses. Interference will naturally result if the two 
responses involve the same effector organ. It is not possible 
to contract the extensors and the flexors of a limb completely 
at the same time. Ordinarily, the interference does not occur 
at this peripheral point, but in the neural mechanism which 
controls the effectors. The stimulus for a flexion reflex in 4 
spinal dog simultaneously inhibits the motor neurons to the 
extensor muscles of the limb. This is the mechanism of recip- 
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rocal innervation which has been so thoroughly studied by 
Sherrington (1906). 

The reciprocal nature of interference is not always as clear 
in the intact animal or human subject. One activity may in- 
hibit another which does not appear to involve the same re- 
sponding mechanism. A loud sound, or clenching the fists, may 
produce no movement of the leg, and yet facilitate the knee 
jerk. In such instances it is not possible to predict accurately 
the interaction effect of one activity upon another in the ab- 
sence of empirical observation. It is generally true, however, 
that a moderate degree of muscular tension throughout the 
body has a facilitating effect upon motor and mental perform- 
ances (G. L. Freeman, 1933), while sudden, novel stimuli 
have a distracting or inhibiting effect. The degree of inhibi- 
tion will also be a function of the intensity of the interfering 
activity (Creed, et al., 1932; Peak, 1936). 

Direct evidence on the summation of conditioned responses 
with reflexes or with other conditioned responses has been ob- 
tained in a few studies (Wendt, 1930; Hilgard, 1931; Garvey, 
1933). With appropriate time intervals, the behavioral result 
of presenting together the stimuli for two responses, each of 
small amplitude, is greater than an algebraic sum of the sep- 
arate components, agreeing with the findings of reflex physi- 
ology. 

Interference involving conditioned responses may be seen 
in several different types of situation. One of the clearest ex- 
amples is the inhibition of a conditioned response by the 
concomitant presentation of another stimulus, described by 
Pavlov as external inhibition. Pavlov has pointed out that 
the most effective stimuli are strong and unusual ones which 
evoke marked investigatory reflexes. Repeated presentation of 
the extra stimulus leads to a progressive loss of the inhibitory 
effect. External inhibition is temporary, lasting from a few 
seconds to several minutes or longer depending on the nature 
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and intensity of the stimulus. Continued inhibition may re- 
sult from persisting stimuli such as those arising from an irri- 
tating injury, distention of the bladder, and sex excitement 
(Pavlov, 1927, p. 47). 

Interaction effects, whether of summation or interference, 


TABLE 3 


EXTERNAL INHIBITION As A FUNCTION OF TEMPORAL INTERVAL 


Conditioned eyelid responses from 16 human subjects, each tested twice 
at each interval. Conditioned stimulus, light; unconditioned stimulus, puff 
of air; extraneous stimulus serving to produce external inhibition, vibrator. 
Conditioning interval between light and air-puff, 500 msec. (Gorham, 
1937.) 





Interval be- 


Interval be- 3 ti 
toten bralo Conditioned responses 


tween vibrator 


and light and conditioned SREQUENCY AMPLITUDE 


es 8 response * — PERCENT —MM. 
—msec. 
Set ee ee T E cele 
Normal 98 21 
(no vibrator) 
Vibrator precedes 
light 300 628 71 18 
100 428 55 I5 
Vibrator follows 
— 100 228 23 3 


as te O ed ee 


* Based on mean normal latency of 328 msec. for the conditioned response. 


are complicated by the temporal relation of the two activities. 
The amount of external inhibition, for example, has been 
shown to depend on the length of interval by which the inhibi- 
tory stimulus precedes the moment of usual occurrence of 
the conditioned response. The relevant data are presented in 
Table 3 (Gorham, 1937). Within the range of temporal inter- 
vals studied, the nearer the presentation of the extra stimulus 
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(vibrator) approached the conditioned response, the greater 
the amount of inhibition. 

Situations in which there is simultaneous presentation of the 
stimuli for two different conditioned responses have received 
almost no experimental study. Sinkevich (1929) trained chil- 
dren by food reinforcement to grasp a rubber bulb with the 
right hand when a red lamp was lighted, and by shock rein- 
forcement to withdraw the left hand when a green bulb was 
lighted. When the two stimuli were subsequently presented at 
the same time, the children reacted in various ways. On some 
trials grasping alone occurred, on others withdrawal alone, 
and on still others both grasping and withdrawal. Although 
the interference situation has not been extensively studied, 
there are many parallels in the conflict of tendencies estab- 
lished by other than conditioning procedures. Results like 
those in Sinkevich’s experiment have been obtained with well 
practised voluntary responses (Hovland and R. R. Sears, 
1938). 

In the situations just described the interference resulted 
from the presentation together of two stimuli having antago- 
nistic response tendencies. Other examples involve the inter- 
ference between two responses evoked by the same stimulus. 
In such cases the incompatible responses cannot be established 
simultaneously, but must be successive. 

To many conditioned stimuli there is already a natural re- 
flex response before conditioning is begun. This response will 
decrease and disappear as learning proceeds if it is antago- 
nistic to the newly developing conditioned response. If no 
antagonism exists between the two responses there will be no 
interference effect. Examples of both results were found in 
some experiments on conditioned eyelid responses of dogs 
(Hilgard and D. G. Marquis, 1935). Before training, the con- 
ditioned light stimulus evoked a short latency (90 msec.) lid 
opening of very small magnitude, and a longer latency (200 
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msec.) partial lid closure. Both responses were bilateral, the 
light having been presented to both eyes. During the acquisi- 
tion of the unilateral conditioned blink reinforced by an air- 
puff to one eye, there was no significant decrement in the origi- 
nal opening response, which did not interact because of its 
short latency. The partial closure response, however, occurred 
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The bilateral closure (Rcs), present before conditioning, gave way during 


conditioning to a unilateral closure (CR) of similar latency. The decrease in 
the bilateral response during conditioning, and increase during extinction, Te- 


flect 2 interference between the two responses (Hilgard and D. G. Marquis, 
1935). 


at about the same latency as the conditioned response, and 
during conditioning was gradually reduced in frequency, to 
return during extinction (Figure Vie 

A similar illustration of interference was given by Pavlov 
(1932b) in his description of the formation of a conditioned 
salivary response to the ringing of a bell. The conditioned 
stimulus originally evoked typical listening movements; the 
dog turned its head and pricked up its ears. The listening 
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movements gradually disappeared as the dog came to respond 
to the bell by licking its chops and turning toward the food 
pan. A more dramatic instance of the same phenomenon was 
afforded by an experiment, also with dogs, in which an electric 
shock was employed as a signal for the presentation of food 
(Eroféeva, 1916). The struggling and yelping elicited by the 
shock were not compatible with the conditioned responses 
which were acquired on the basis of food reinforcement, and 
the original responses gradually diminished until after many 
training sessions almost no trace of them remained. 

If the original response is a previously established condi- 
tioned response, interference will result from the formation of 
a new incompatible response to the same stimulus. This pro- 
cedure may be termed counter conditioning; it is the familiar 
one of breaking a habit by learning to do something else. The 
transition from a nocuous to a food conditioned response has 
been studied by Slutskaya (1928) in young children. The 
subjects first formed a conditioned avoidance response to the 
sight of a needle with which they were pricked. In later trials 
the pricking was always followed immediately by feeding, 
and in five of the eight children the avoidance response gradu- 
ally weakened and disappeared, so that finally the sight of 
the needle occasionally evoked mouth opening and swallow- 
ing movements. The drug addict’s craving for the hypodermic 
needle is a familiar parallel. 

More detailed study of counter conditioning has been re- 
cently undertaken by Kellogg and E. L. Walker (1938a). Dogs 
were trained to lift the right leg to a conditioned stimulus 
(buzzer) by shocking the right foot. Then the electrodes for 
shocking were transferred to the left foot, and conditioning of 
the left leg response to the same buzzer stimulus was under- 
taken. The initial effect of the new situation was to intensify 
the conditioned response of the right leg. In the particular 
apparatus used the dog was able to lift both legs together by 
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supporting its weight on a wooden stock enclosing its neck, 
and this double response was common, persisting for as many 
as 100 trials. Eventually the right leg response disappeared 
and the dog responded only with the left foot. 

An original response is inhibited only as long as the inter- 
fering response remains active, and subsequently shows little 
or no residual impairment. This is clear in the case of external 
inhibition; a few seconds after the extra stimulus has ceased 
the conditioned response may be evoked at full strength. The 
inhibition of an original response to a conditioned stimulus 
likewise persists only as long as the conditioned response in- 
terferes. When the latter is reduced by extinction, the original 
response reappears (Figure 7). Hull (1934b) has described a 
corresponding finding in an instance of counter conditioning. 
Rats were first trained to run down a 20 foot straight alley 
to receive food. When the alley was lengthened to 40 feet, 
they showed a tendency to halt at the 20 foot point, but with 
continued training this tendency disappeared and the animals 
ran steadily down the entire path. During extinction of the 
later habit by omission of food reinforcement at the end of the 
alley, the tendency to halt at the 20 foot point again reap- 
peared. A somewhat comparable finding is reported by Poz- 
nanskaya (1934) with children who were trained to give a 
new conditioned response to the same stimulus after the first 
conditioned response was extinguished. When the substituted 
response was extinguished, the earlier conditioned response 
reappeared. Further evidence for the preservation of the orig- 
inal response would follow from the demonstration of its re- 
appearance when the second response was inhibited by an 
extra stimulus. Several experimenters have shown that an 
animal will regress to an earlier habit when strong emotional 
stimuli are presented (J. A. Hamilton and Krechevsky, 1933; 
Everall, 1935; N. E. Miller and Stevenson, 1936; Sanders, 
1937). The possible relation between this finding and the 
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Freudian theory of regression has been pointed out by sev- 
eral writers. 

The general characteristics of the inhibition resulting from 
interference may be summarized as follows: 

1. Interference occurs when two incompatible responses are 
evoked at the same time, either by the same stimulus or by 
separate stimuli. 

2. The degree of inhibition is a function of the nature and 
intensity of the interfering response, and the temporal rela- 
tion of the two responses. 

3. Inhibition, which is a suppression of response and not 
a permanent destruction, persists as long as the interfering 
activity is present. Recovery from inhibition will not be spon- 
taneous, but will depend upon the removal of the interfering 
activity by extinction, forgetting, or externa! inhibition. 


The Interpretation of Extinction 


The term ‘experimental extinction’ was used by Pavlov to 
designate the decrement in a conditioned response during 
nonreinforced repetition expressly because it did not imply 
any theory of the mechanism of decrement. Extinction, there- 
fore, properly refers to a procedure rather than to a process. 
It may involve either (or both) adaptation and interference. 

Pavlov’s theory of extinction is stated in terms of internal 
inhibition, which is essentially the same as the mechanism of 
adaptation described above. Although Pavlov’s explanation 
involved the neural processes of excitation and inhibition, it is 
possible to ignore the speculative neurological explanation 
and treat extinction as a change in the relationships between 
stimuli and responses. The essential theory is that the decre- 
ment during extinction is due to a cumulative unresponsive- 
ness resulting from the repeated elicitation of the response it- 
self. 
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As an alternative to Pavlov’s explanation of extinction, one 
stated in terms of interference has been proposed by Guthrie 
(1935) and by Wendt (1936). According to this theory, the 
decrement in the conditioned response is due to the inter- 
ference of some new response to the conditioned stimulus 
which is gradually strengthened during the successive non- 
reinforced repetitions, Extinction thus becomes the same proc- 
ess as counter conditioning. This theory has the attractive 
advantage that it explains extinction without the necessity 
of adding a new principle of adaptation to the two basic prin- 
ciples of reinforcement and interference. In order to weigh 
the accuracy of the explanation three points demand discus- 
sion. 

The first point concerns the nature and source of the inter- 
fering response. Careful observation often reveals that the 
subject is not merely withholding the conditioned response, 
but is instead actively doing something else. During extinction 
of conditioned feeding responses, infants may show avoidance 
of the empty nursing bottle by making pushing gestures and 
by turning the head away (Levikova and Nevymakova, 1929; 
Denisova and Figurin, 1929; Kantrow, 1937). Monkeys 
trained to open a drawer on signal to obtain food show during 
extinction an increase of other activities such as vocalization, 
biting and pulling the drawer, and racing about the cage 
(Wendt, 1936). In most extinction experiments, the presence 
of new antagonistic activity has not been noted, partly, of 
course, because the experimenter’s attention has been directed 
chiefly to the conditioned response itself. From other experi- 
ments (eg., Zener, 1937), it is clear that new responses during 
extinction may be original responses to the conditioned stim- 
ulus which were inhibited by the conditioning, responses te 
other features of the total experimental situation, or re- 


sponses generalized from previous experience with similar 
non-reinforcing or frustrating situations, 
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The second point concerns the possibility of successful in- 
terference by the new responses. Their presence during ex- 
tinction is not itself convincing evidence for the interference 
theory. It may be more plausible to consider them as effects 
of extinction than as causes. Sleep, for example, may result 
from generalization of inhibition (Pavlov), or it may be itself 
a reaction incompatible with conditioning (Wendt). In in- 
stances of counter conditioning, the new response is specifi- 
cally reinforced so that it becomes strong enough to inhibit the 
original conditioned response. Where such reinforcement is 
not present, the occurrence of new responses will be possible 
only after the conditioned response has been weakened suf- 
ficiently to permit their appearance. A conditioned response 
may be weakened without adaptation or interference, as, for 
example, by a reduction in the intensity of the conditioned 
stimulus, by fatigue of the conditioned response, by external 
inhibition, or by attempted elicitation during the refractory 
period of a previous response (Guthrie, 1935). Some such 
weakening process must be inferred if antagonistic responses 
are to reappear, unless a mechanism is suggested for strength- 
ening the interfering responses. The interference interpreta- 
tion of extinction, if it is not to imply adaptation or other 
weakening independent of interference, must discover a source 
for strengthening those responses which are antagonistic to 
the conditioned response. 

This leads to the third point: can the course of extinction, 
and its other functional characteristics, be accounted for on 
the basis of an interference theory? Many of the experimental 
facts fit such a theory very well. The progressive, cumulative 
course of extinction would be an expression of the degree of 
learning of the incompatible response, and extinction below 
zero would be the result of overlearning. Disinhibition would 
be due to an external inhibition of the competing response. 
There are, however, three characteristics of extinction which 
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are difficult to reconcile with the interference theory. These 
are spontaneous recovery, the effect of rate of elicitation, and 
the effect of drugs. 

Spontaneous recovery, according to the interference theory, 
could occur only to the extent that the competing re- 
sponse is weakened during the interval by forgetting. Experi- 
mental data, however, indicate that spontaneous recovery 1s 
much more rapid, being measured ordinarily in terms of 
hours, whereas forgetting is measured in terms of days or 
months and in many cases is not appreciable over even longer 
periods. If extinction occurs through interference by a learned 
response, the spontaneous recovery would be expected to be 
just as slow as the forgetting of this newly acquired antagonis- 
tic response. Extinction by counter conditioning, in which the 
operation of interference is clear, does fail to show rapid spon- 
taneous recovery, according to the fragmentary evidence now 
available. Kellogg and Wolf (1939) found that after intervals 
up to three months there was no recovery of responses inhib- 
ited by counter conditioning, and that the new interfering re- 
sponse was correspondingly undiminished. The spontaneous 
recovery following extinction by punishment for response has 
not been tested, but subsequent reconditioning was found to 
require more training than after simple extinction (Brogden, 
Lipman, and Culler, 1938). 

; A second line of evidence against the interpretation of ex- 
tnction as interference from new learning is derived from 
the study of the effect of rate of elicitation. Massing of trials 
tends to result in more rapid extinction, but the speed of con- 
ditioning is retarded under similar circumstances (see p. 133). 
If the extinction process were essentially similar to the con- 
ditioning process, one would expect a positive correlation 
between the rates of acquisition and extinction, that is, rapid 
conditioning corresponding to rapid extinction, slow condi- 
toning corresponding to slow extinction. The numerous corre- 
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lations reported are, however, predominantly negative. Some 
of the correlations based on populations large enough to sug- 
gest significance are summarized in Table 4. 


TABLE 4 


REPRESENTATIVE CORRELATIONS Berween RATE oF CONDITIONING, AND 
Rate or EXTINCTION 


The signs of the correlation coefficients have been arranged so that a 
negative sign means that rapid or more complete conditioning is associated 
with slow or less complete extinction. 





Subjects, uncon- Measures Number Coe ficient 
ditioned stimuli and of conditioning of of 
responses; experimenter and extinction subjects correlation 
Rats; running to shock; Trials to 10 successive 
Hunter (1935a) CR’s; trials to 5 successive r=—.36 
failures to respond 5I n=- .66 
Children; movements asso- Trials for conditioning; 
ciated with eating; Mateer trials for extinction 50 r=—.45 
(1918) 


College students; eyelid Frequency of CR within 50 
response to electric shock; reinforcements; number of 
Bernstein (1934) CR’s during extinction 69 =- .88 


College students; eyelid Frequency of CR within 50 
response to air-puff; reinforcements; frequency 
A. A. Campbell (1938) within 10 extinction trials 49 r=—.66 


College students; knee jerk Frequency of CR within 100 
to tap on patellar tendon; reinforcements; frequency 
A. A. Campbell (1938) within I0 extinction trials 49 r=-—.60 








Further evidence for the functional dissimilarity of extinc- 
tion and conditioning is derived from the differential effects 
of certain drugs. Sodium bromide and other drugs of a de- 
pressant character retard the rate of conditioning, but ac- 
celerate the rate of extinction. Excitants such as caffeine and 
benzedrine have been found to increase the strength of con- 
ditioned responses but to decrease the rate of extinction. It 
might be inferred that such drug effects are due merely to a 
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change in the general level of reactivity of the organism which 
would inevitably result in larger conditioned responses and 
consequently delay extinction. Switzer (1935a) found that 
this was the result when caffeine was administered to human 
subjects in an experiment on conditioned galvanic responses. 
However, this general increase was not sufficient to account 
for the full extent of the retardation of extinction. When the 
initial level of responding was equated for both the control 
and the experimental (caffeine) groups of subjects, there still 
remained a significantly more rapid rate of extinction in the 
former. Pavlov, too, was unwilling to accept an explanation 
in terms of changed level of reactivity. In a dog that had gone 
into a state of experimental neurosis he administered sodium 
bromide and observed a reappearance of extinction effects 
previously impossible. At the same time there was no diminu- 
tion in the magnitude of the positive conditioned responses. 

From the above considerations it becomes evident that a 
theory of extinction in terms of interference between the con- 
ditioned response and newly acquired responses is not suf- 
ficient to account for all the experimental facts. Interference 
will, of course, be a factor in extinction at least to the extent 
that reinforcement for new incompatible responses is present 

in the situation. Adaptation appears to be necessary to account 
for the rapidity of spontaneous recovery, the more rapid ex- 
tinction with massed repetitions, and the differential effect of 
certain drugs upon extinction and conditioning. At the same 
time it is clear that the concept of interference is necessary to 
account for the differences between simple extinction and 
counter conditioning. Both processes are found in most in- 
Stances of extinction. 

A theory of extinction must be adequate to account for in- 
stances in which the expected decrement does not occur. Ac- 
cording to the adaptation theory, extinction must inevitably 
result from repetition of the conditioned response unless some 
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maintaining influence is strong enough to overcome the ef- 
fect. Interference will produce extinction whenever a stronger 
incompatible response is elicited. Several examples of ap- 
parent failure of extinction have been reported, although the 
responsible factor is seldom revealed by the experiments. 
Many instances can best be understood as the continued 
operation of reinforcement. This is clearly illustrated in the 
extinction of responses based on derived reinforcement. A 
dog which has been trained to lift its foot to escape a shock to 
the paw may show no extinction when the conditioned stimu- 
lus is repeatedly presented without the shock. But in this case, 
as was pointed out earlier, the shock itself is not the reinforce- 
ment; instead the reinforcement is based on absence of ex- 
pected shock. Continued omission of the shock will therefore 
not lead to extinction. There is no procedure by which the 
derived reinforcement can be omitted. Only if the dog should 
fail to lift his leg sometime for any reason would the absence 
of the shock weaken the expectancy and lead to extinction. 
This type of situation may perhaps be typical of habits which 
persist when no apparent reinforcement follows the act. The 
fact that “a burnt child shuns the flame” does not require that 
the child be burned every time he sees the candle; the avoid- 
ance response is strengthened rather than weakened when- 
ever the child escapes burning. 

When conditioned responses are established with reinforce- 
ment presented on only a fraction of the training trials, ex- 
tinction tends to be slower. This situation has been studied 
by Skinner (1938), and Humphreys (1939a, 1940c) with con- 
cordant results for different conditioned responses in rats and 
human subjects. These experiments are discussed in more 
detail in a later chapter (p. 150), and an interpretation is 
suggested in terms of expectancy. Prior to the beginning of 
extinction, conditioning by substitution or by effect had been 
replaced by expectancy of occasional feeding or occasional 


122 CONDITIONING AND LEARNING 


air-puffs. Since the training procedure maintained this ex- 
pectancy in spite of occasional omission of the unconditioned 
stimulus or the reward, the consistent omission during extinc- 
tion did not produce as rapid a destruction of the expectancy 
as occurred during extinction in subjects which had received 
reinforcement on every training trial. A similar explanation 
is perhaps appropriate for the failure of extinction of con- 
ditioned lid responses in monkeys (Hilgard and D. G. Mar- 
quis, 1936). This result is puzzling because directly compar- 
able studies in dogs and in human subjects demonstrated 
typical extinction. The monkeys differed from the other sub- 
jects,. however, because they closed their eyes completely 
rather than partially on every occurrence of the conditioned 
response. The air-puff was therefore received on the eyelid 
rather than the cornea and presumably was less intense. This 
would make the conditioning more similar to instrumental 
avoidance or to the occasional reinforcement training proce- 
dure, either of which leads to very slow extinction. 

In other examples of the failure of extinction, an expectancy 
may be established by self-instructions and this takes the 
place of the original reinforcement which is now omitted. 
This would be most likely to occur in human subjects. Razran 
(1935), for example, failed to find consistent extinction of 
salivary conditioned responses. In a later experiment, he 
avoided the possibility of subjective attitudes and association 
sets by misinforming his subjects concerning the experiment 
(19391). Under these conditions, extinction was readily dem- 
onstrated. Several investigators have reported aberrant re- 
sults in attempting to extinguish conditioned finger withdrawal 
in human subjects. It seems that the more completely the 
response is capable of control by verbal voluntary processes, 
the greater is the possibility that extinction will fail. A con- 
ditioned response reinforced by a purely voluntary act may 
continue undiminished when the overt reinforcement is dis- 
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continued (D. G. Marquis and Porter, 1939). This function 
of verbal reinforcement has been demonstrated in an experi- 
ment by Hudgins (1933) who conditioned pupillary constric- 
tion reinforced by a bright light to a series of different con- 
ditioned stimuli. Extinction was obtained normally in the 
response to external stimuli such as a buzzer or hand squeez- 
ing, but when the response was finally established to implicit 
self-spoken commands to “Contract,” extinction no longer 
occurred. A somewhat similar result for vasomotor responses 
was reported by Menzies (1937). 

An apparent paradox in the extinction of voluntary re- 
sponses is readily resolved if the expectancy principle is ac- 
cepted. Voluntary responses may be strengthened by derived 
reinforcement, and hence may not show usual extinction. On 
the other hand, being under the influence of expectancy, they 
may under some circumstances show unusually rapid extinc- 
tion, for counter conditioning may also be reinforced indi- 
rectly. Thus Porter (1938d) found in his study of the extinc- 
tion of conditioned eyelid responses in man that subjects either 
extinguished very rapidly (in a mean of 6.8 = 1.0 trials fol- 
lowing 75 reinforcements) or they tended to show no ex- 
tinction at all in several periods of 15 non-reinforced trials. 
Reports from the subjects make it possible to refer the 
differences to unlike sets or expectations. Rapid extinction 
may be a result of a tendency to cease responding which is 
reinforced verbally, quite as much as failure to extinguish in 
some cases depends upon continued responding as a precau- 
tion against the re-introduction of the unconditioned stimulus. 

There are cases of failure of extinction which offer no clear 
clues to an explanation in terms of continued derived rein- 
forcement. Harlow (1937) found no extinction of conditioned 
startle response in monkeys to a signal which had been fol- 
lowed by the report of a popgun. Baker (1938) has reported 
that the conditioned pupillary response to subthreshold stim- 
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uli undergoes no decrement with unreinforced repetition. Re- 
tention was also perfect over many months, and the only way 
the response could be eliminated was by counter condition- 
ing. That is, pupillary dilatation was extinguished by follow- 
ing the conditioned stimulus with a bright light, thus estab- 
lishing conditioned constriction. Confirmation of these results 
will constitute a strong argument for extinction by interfer- 
ence. 


Inhibition in Situations 
Other than Extinction 


Although extinction has been the subject of more experi- 
mental study than other situations involving inhibition it is 
by no means the only procedure within conditioning for the 
establishment of inhibition. The omission of reinforcement 
is not the essential condition for inhibition, but merely reveals 
adaptation and interference in unopposed form. A tendency 
to adaptation should be found whenever a response is repeat- 
edly elicited, and interference should be found whenever there 
is a possibility of another antagonistic response being simul- 
taneously evoked. Some of these situations will be described. 

Inhibition During Continued Reinforcement. In sev- 
eral experiments it has been found that a decrement in the 
conditioned response may occur in spite of continued rein- 
forcement, when the successive training trials are presented 
in rapid succession (e.g., Wendt, 1930; Hilgard, 1933a; Kan- 
trow, 1937). If a rest interval is interpolated in the learning 
series, spontaneous recovery from the adaptation occurs, and 
following the interval the conditioned response is stronger 
than before it. This has been shown for overnight effects in 
dog, monkey, and man (Hilgard and D. G. Marquis, 1935, 
1936; Hilgard and A. A. Campbell, 1936). Adaptation which 
IS not great enough to cause an actual decline in the condi- 
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tioned response but merely retards the rate of acquisition can 
be demonstrated by the augmenting influence of a rest period 
or an extraneous stimulus. 

Inhibition by Punishment. If a response is followed by 
a noxious stimulus it will tend to be weakened in strength and 
less likely to occur on the next presentation of the situation. 
This type of procedure has been frequently employed in in- 
strumental training, particularly in discrimination problems, 
but has not been extensively studied in classical conditioning 
situations. Brogden, Lipman and Culler (1938) have demon- 
strated that extinction of conditioned leg retraction in the dog 
is much more rapid if an electric shock on the chest is given 
when the dog reacts to the conditioned stimulus and none 
when it fails to respond. 

Although it was part of his original statement of the law 
of effect, Thorndike (1932a, 1932b) has subsequently ques- 
tioned the existence of inhibition by punishment. On the basis 
of extensive experiments with chickens and with human sub- 
jects he has concluded that an annoying after-effect has no 
weakening effect comparable to the strengthening which re- 
sults from a satisfying after-effect. When responses appear to 
be weakened by punishment it is only indirectly through 
strengthening the alternative competing responses. This rather 
startling conclusion has become the subject of a vigorous con- 
troversy. W. Brown (1939) designed an experiment which 
demonstrated the positive effect of punishment on the choice 
behavior of rats in a single unit T-maze, and questioned the 
appropriateness of the experiments which had yielded nega- 
tive results. Tilton (1939) reviewed the published evidence on 
the multiple-choice experiments with human subjects, and 
pointed out that it is not justifiable to measure the effect of 
“Right” and “Wrong” from a base-line of calculated chance 
repetition. When allowance was made for the natural tendency 
to repeat the responses given on the first trial, the results 
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showed that punishment (“Wrong”) had a definite weaken- 
ing effect. 

The action of punishment is a subject urgently needing ad- 
ditional research. It is not clear at present whether inhibition 
by punishment can be accounted for in terms of interference 
from responses established by the noxious stimulus or whether 
it represents a different and independent form of inhibition. 

Inhibition by Discrimination. In the usual method of 
contrasts, in which one of a pair of stimuli is reinforced, and the 
other non-reinforced, the diminution of response to the non- 
reinforced stimulus is somewhat comparable to extinction, 
although the processes may differ in detail (Hilgard, R. K. 
Campbell, and W. N. Sears, 1938). The rate of decrease is 
complicated by some generalization from the positive stimu- 

lus. In some cases the contrast is made more explicit by re- 
warding response to the positive stimulus and punishing re- 
sponse to the negative stimulus (W. T. James, 1933). While 
the diminution of response is slower than in ordinary extinc- 
tion spontaneous recovery also appears to be less, so that 
discrimination may persist for months close to its original 
value (Hilgard and Humphreys, 1938b). 

Conditioned Inhibition. Multiple stimuli may be used 
in a discrimination situation, so that a combination of stimuli 
Sı + Sə will be rendered negative through non-reinforcement, 
while S} remains positive. In this case, Se is called the condi- 
tioned inhibitor, and other conditioned responses tend to be 
diminished in its presence (Pavlov, 1927; Lindberg, 1933): 
The problems of discrimination are discussed more fully in 
Chapter 8. í 
| Inhibition of Delay. The period of reduced response dur- 
ing the interval between the beginning of the conditioned 
stimulus and the appearance of a trace or delayed conditioned 

response has many characteristics of extinction. There is 
spontaneous recovery, so that the delay is reduced after lapse 
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of time (Switzer, 1934; Rodnick, 1937a); there may be dis- 
inhibition by an extraneous stimulus; and other concomitant 
conditioned responses may be reduced if they are elicited dur- 
ing the delay period (Rodnick, 1937b). 

Forgetting. Forgetting differs from the other situations 
described in that the response is presumed not to be elicited 
during the interval over which retention is measured, and 
hence adaptation cannot be the primary interpretation. It is 
possible, however, that similar conditioned stimuli occur dur- 
ing the retention interval, evoking the conditioned response 
(or portions of it), so that adaptation cannot be completely 
ruled out. In view of the accumulating evidence favoring 
an interpretation of forgetting through retroactive inhibition 
(Britt, 1936; McGeoch, 1932), it appears that interference 
may be the more important factor. Competing responses are 
learned during the retention interval. The dependence of de- 
gree of forgetting upon the duration of the interval would thus 
be really a dependence upon the number and strength of in- 
terfering tendencies formed during that time. The resistance 
of laboratory conditioned responses to forgetting may be due 
to the highly artificial circumstances of stimulation in the 
laboratory, so that few competing responses are learned dur- 
ing the interval that the subject is outside that situation. 

Disinhibition. When an extraneous stimulus reverses the 
process of extinction, so that temporarily the conditioned stim- 
ulus again evokes its conditioned response at approximately 
normal magnitude, the effect is known as disinhibition. This 
is the counterpart of external inhibition. In the one case the 
external stimulus reduces a conditioned response in the course 
of acquisition, in the other it increases the response in course 
of extinction. In both cases the effect is temporary, and after 
lapse of a short time the conditioned response appears at ap- 
proximately its magnitude prior to the intrusion of the extra 
stimulus. 
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Two possibilities of interpretation of disinhibition are sug- 
gested by our preceding analysis. As a first possibility, the 
extra stimulus may have a facilitating influence, and serve to 
increase the response through summation. This interpretation 
appears to be applicable to such cases as the disinhibition of 
adapted reflexes (Humphrey, 1933). As a second possibility, 
the extra stimulus may interfere through external inhibition 
with a process which is itself interrupting the conditioning. 
The apparent paradox that an extraneous stimulus decreases 
a response in course of acquisition and increases one in course 
of diminution is in part resolved by considering the extra 
stimulus as a distraction, temporarily reducing the effect of 
whatever process is prominent at the time. 


Summary 


Inhibition is defined as the decrement in response which | 
results from stimulation. Two varieties are distinguished, in- 
trinsic inhibition or adaptation, resulting from the repetition 
of the response itself, and extrinsic inhibition or interference, 
resulting from the appearance of antagonistic responses. These 
processes include more than experimental extinction, and ap- 
ply to decrements in reflexes and voluntary responses as well 
as to conditioned responses. Experimental extinction is de- 
fined as the diminution of the strength of conditioned re- 
sponses by the repeated presentation of the conditioned stimu- 
lus without reinforcement. It refers to a procedure rather than 
to a process. Both adaptation and interference are necessary 
to explain extinction. Adaptation of conditioned responses is 
comparable to the habituation of reflexes or to the work decte- 
ment found in repeated voluntary acts. Evidence for inter- 
ference comes most clearly from counter conditioning, 10 
which old learning is antagonized by new. Cases in which ex- 
perimental extinciion does not occur with usual rapidity often 
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arise in situations in which the effects of adaptation and inter- 
ference are counteracted by derived reinforcement, which 
continues although the original unconditioned stimulus, re- 
ward, or punishment, is omitted. 

There are many other procedures leading to response inhi- 
bition, such as discrimination, delay, forgetting. These show 
functional similarities, in that adaptation and interference are 
important in all. More experimental evidence is needed before 
a precise formulation may be made of the relative significance 
of adaptation and interference in inhibition. 


Notes 


Extinction an Active Process 


That extinction is an active sup- 
pression of response, and not a mere 
fading is shown in the long retention 
of conditioned responses. A number 
of earlier studies are summarized 
by Razran (1933a, 1933b). Among 
more recent reports may be men- 
tioned the retention of conditioned 
motor responses by sheep for 2 
years (Liddell, W. T. James, and 
O. D. Anderson, 1934); conditioned 
eyelid reactions in dogs for 16 
months (D. G. Marquis and Hil- 
gard, 1936); conditioned eyelid re- 
actions in man for 20 weeks (Hil- 
gard and A. A. Campbell, 1936) 
and for 19 months (Hilgard and 
Humphreys, 1938b); conditioned 
pupillary responses in man for 2 
years (Baker, 1938); conditioned 
flexion reflex in the dog for 24% years 
(Wendt, 1937); conditioned saliva- 
tion in man for 16 weeks (Razran, 
1939f); various responses in dogs for 
6 months (Kellogg and Wolf, 1939). 

The enduring character of condi- 


tioned responses, and the temporary 
character of extinction, have led 
some writers to separate characteri- 
zations of ‘excitatory’ and ‘inhibito- 
ry’ response tendencies, the inhibi- 
tory tendencies appearing to be 
more labile than the excitatory ones. 
An assumption of this sort appears 
to be necessary to account for spon- 
taneous recovery following extinc- 
tion. 

It is a mistake, however, to over- 
simplify the interpretation of the re- 
sults of extinction. The facts of rap- 
id extinction and rapid recovery do 
not mean that the results of extinc- 
tion are solely temporary. An illus- 
tration of the endurance of conse- 
quences of extinction is found in 
Keller’s results for instrumental re- 
sponses, in which extinction after an 
interval of 45 days showed the dele- 
terious effects of earlier extinction, 
in spite of spontaneous recovery 
(Skinner, 1938, 94-96). As Skinner 
points out, it cannot be concluded 
that the animals have ‘forgotten the 
extinction while remembering the 
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conditioning.’ Another illustration 
of the endurance of inhibition in a 
conditioning situation is found in the 
retention of discrimination. It has 
been shown that conditioned dis- 
criminations may be retained over a 
year and a half (Hilgard and Hum- 
phreys, 1938b). Although the tend- 
ency to respond to the positive stim- 
ulus survived the interval somewhat 
better than the tendency to inhibit 
response to the negative stimulus, 
the inhibitory tendencies were not 
lost over this period of time. 


Intrinsic and Extrinsic Inhibi- 
tion in Relation to 
Pavlov’s Concepts 


Intrinsic inhibition or adaptation 
corresponds approximately to the 
behavioral meaning of Pavlov’s in- 
ternal inhibition, while extrinsic in- 
hibition or interference corresponds 
in part to his external inhibition. 
External inhibition, according to 
Pavlov, referred exclusively to a 
temporary disruption; extrinsic in- 
hibition, as we are using it, may re- 
fer to relatively enduring disruptions 
due to new habit acquisitions, and 
hence may explain some of the phe- 
nomena which Pavlov would include 
under internal inhibition. 


Adaptation 


The habituation or negative adap- 
tation of reflexes is treated inciden- 
tally in many studies of reflex ac- 
tion. The parallels with conditioning 
phenomena are best brought out by 
Humphrey (1930, 1933), who gives 
evidence from his own experiments 
with snails and turtles. The coordi- 
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nation with internal inhibition is 
made by Oldfield (1937). The rela- 
tion of reflex habituation to learning 
is discussed, with experimental evi- 
dence, by Dodge (1923), Halstead 
(1935), Halstead, Yacorzynski, and 
Fearing (1937), O. H. Mowrer 
(1934) all of whom are concerned 
with habituation to rotation. The 
earlier literature is reviewed in Fear- 
ing’s history of reflex action (1930). 

Examples of ‘pure’ adaptation are 
as rare as examples of ‘pure’ substi- 
tution. The wavelike character of 
curves of extinction (Arakelian, 
1939) suggests complications be- 
yond simple cumulative decrement. 
Skinner (1938) attributes the ir- 
regularities to emotional disturb- 
ances. Although N. E. Miller and 
Stevenson (1936) noted the pres- 
ence of overt agitation during the 
extinction of an instrumental loco- 
motor response in rats, they were 
unable to correlate the occurrence of 
agitation with cycles in the curve of 
extinction. 

Adaptation is superficially similar 
to fatigue. However, if fatigue is de- 
fined as a decrement resulting from 
the exhaustion of energy supplies or 
the accumulation of end products of 
metabolism in the component parts 
of the responding mechanism, 1t 18 
evident that adaptation is not fa- 
tigue. Disinhibition shows that un- 
der favorable circumstances the ex- 
tinguished response can again be 
evoked, without waiting for meta- 
bolic restoration. Furthermore, 4 
rate of stimulation which is rapid 
enough to produce adaptation may 
be too slow to cause fatigue. __ 

When one conditioned response 18 
extinguished, other conditioned re- 
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sponses likewise undergo extinction. 
This is known as secondary extinc- 
tion (Ellson, 1938; Hill and Calvin, 
1939; Youtz, 1939). Generalization 
of extinction has been studied ex- 
tensively in so-called irradiation ex- 
periments (Anrep, 1923; Bass and 
Hull, 1934; Hovland, 1937a). These 
are discussed more fully in Chap- 
ter 8. 

The generalization of inhibition 
appeared so prominently in Pav- 
lov’s experiments that he identified 
sleep and hypnosis with it. His the- 
ory has not been widely accepted 
because of lack of direct evidence. 
The conception has been discussed 
and extended by Anokhin (1932), 
Krasnogorsky (1923), Rojansky 
(1914), M. Levin (1934a, 1934b), 
S. L. Levin (1934). 


Interference 


The belief that elimination of a 
learned act depends on its replace- 
ment by a new act has had a long 
history. The recent setting of it in 
opposition to Pavlov’s explanation 
of extinction has been shared by 
several writers, among them Guth- 
rie (1935), Wendt (1936) and Cul- 
ler (1938b). 

The importance of time interval 
in the interaction of acts at a reflex 
level has been demonstrated in a va- 
riety of experiments on facilitation 
and inhibition, e.g., Bowditch and S. 
W. Warren (1890), Yerkes (1905), 
Wendt (1930), Eccles and Sher- 
rington (1931), Hilgard (1933b), 
Misbach (1937). The facilitation 
and inhibition in these studies is of 
a reflex response, although in some 
of them the interacting response is 
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voluntary. Studies in which the mod- 
ified response is verbal or voluntary 
include Hofbauer (1897), Allison 
(1932), Telford (1931), Telford and 
B. O. Anderson (1932), Telford and 
N. Thompson (1933). 

Counter conditioning is a com- 
monplace in instrumental discrimina- 
tion situations in which the positive 
and negative stimuli are presented 
simultaneously and one response is 
punished, the other rewarded. The 
mechanism is exhibited most clearly 
in situations requiring a reversal of 
a discrimination (e.g., Bunch, 1939; 
Buytendijk, 1930; Fritz, 1930; Kre- 
chevsky, 1932b; Hull and Spence, 
1938). The ‘ambiguous condition- 
ing’ experiments in which counter 
conditioning is involved are reported 
by Kellogg (1939a), Kellogg and E. 
L. Walker (1938a, 1938b), Kellogg 
and Wolf (1939). 

External inhibition, which pro- 
vides some evidence for interference, 
is reported by Gorham (1937), Raz- 
ran (1939g), Wenger (1936a), Yush- 
chenko (1928). The lack of success 
reported by A. R. Miller (1934) is 
accounted for by Lindberg (1936) as 
a result of failure to fulfill the many 
conditions necessary for its satisfac- 
tory demonstration. 


The Interpretation of Extinc- 
tion 


The most thorough review of the 
facts and theories of extinction is 
that of Razran (1939)). He sum- 
marizes a large number of the be- 
havioral facts of extinction in order 
to criticize proposed theories, and 
offers a synthetic hypothesis. Among 
the chief theories summarized by 
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Razran are: internal inhibition 
(Pavlov, 1927); adaptation plus 


reverse conditioning (Beritov, 1927, 
1932); change in attitude or af- 
fectivity (Guthrie, 1935; Winsor, 
1930a, 1930b); competing behavior 
systems (Wendt, 1936); change in 
motivation (Finch and Culler, 1934; 
Culler, 1938b); muscular relaxation, 
supplementing specific interference 
(Wenger, 1937); asynchronization 
of nervous discharges (Razran, 
1930). Razran’s present views on 
the behavioral phenomena of ex- 
tinction may be summarized in his 
own words: “(1) ‘decrement by rep- 
etition’ is the most universal factor 
of extinction, a factor which extinc- 
tion shares in some way with habitu- 
ation and adaptation; (2) reverse 
conditioning—restoration of the 
original response to the conditioned 
stimulus—is the general contribut- 
ing factor of extinction, which dis- 
tinguishes extinction from habitua- 
tion and from adaptation; (3) re- 
directed conditioning—conditioning 
to some action arising in the course 
of extinction—is a contributing fac- 
tor that often but not always modi- 
fies the extinctive process; (4) 
changes in affectivity, attitude, 
muscular tension, motivation, and 
the like are only modifying organ- 
ismic states and are not constitutive 
of the extinctive process per se.” 
(Razran, 1939j, 270-271.) 

Apart from matters of emphasis, 
there is considerable harmony be- 
tween the views expressed in the 
text and those of Razran as stated 
above. The acceptance of both 
adaptation and interference is a 
fundamental agreement. 

An ingenious explanation of spon- 
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taneous recovery consonant with an 
interference theory has been pro- 
posed by N. E. Miller and Miles 
(1936), and N. E. Miller and Ste- 
venson (1936). The explanation de- 
pends on the more rapid forgetting 
of the less well learned interfering 
act in competition with the forget- 
ting of the previously acquired con- 
ditioned response. The theory de- 
mands a rapid forgetting of both 
conditioned responses, however, 
which is not in harmony with the 
relative permanence of condition- 
ing. 

While it is true that spontaneous 
recovery, rate of elicitation, and 
drug effects tend to differentiate the 
functional processes of adaptation 
and interference there is a subordi- 
nate aspect of the interference the- 
ory which is compatible with certain 
of these phenomena. One aspect of 
interference, stressed by Winsor 
(1930b), and espoused in part by 
Guthrie (1935), attributes the in- 
terference to altered attitudes and 
emotional states present temporarily 
as a result of omitted reinforce- 
ment. Even relatively short lapses 
of time may restore equanimity Or 
alter expectancies, and account for 
the rapid ‘forgetting’ of the inter- 
fering activity required by sponta- 
neous recovery. Attitudes and sets 
may also be temporarily disrupted 
by distracting stimuli, so that dis- 
inhibition is not excluded. Frus- 
tration may be exaggerated in the 
massed non-reinforcement situation. 
Hence the very arguments which 
most strongly support adaptation 
may also be used to favor a form 
of interference theory based on 
expectancy rather than upon the 
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gradual conditioning of incompatible 
responses. The confident interpre- 
tation of any one illustration of ex- 
tinction can be made only in the 
light of a detailed analysis of the 
experimental situation and the 
particular concomitant activity in- 
volved. Theories of attitude and set 
relative to extinction have not been 
very thoroughly worked out; they 
obviously fit in the category of 
counter conditioning based on de- 
rived reinforcement. Symbolic ac- 
tivity as it enters conditioning is 
treated more fully later (Chapters 
10 and 11). 

By defining extinction in terms 
of rate of elicitation and reduction 
of reflex reserve, Skinner (1938) is 
able to differentiate between a tem- 
porary interruption in responding 
(an ‘emotion’ by definition in his 
system) and an unlearning which 
reduces the number of responses in 
the reserve. He has determined, for 
example, that following a slap upon 
pressing the lever the rats re- 
sponses show ‘emotion’ rather than 
extinction by interference or counter 
conditioning (Skinner, 1938, p. 
155). Within his system, therefore, 
some of the things attributed by us 
to extinction by interference will 
not be included within extinction. 
We agree that any instance of ex- 
tinction requires careful analysis in 
order to determine what specific 
processes are involved, but we are 
unable to apply Skinner’s differen- 
tiations to the range of other experi- 
ments in which extinction is found. 
The analysis within his own situa- 
tion is specific to circumstances 
of experimentation. Thus, non-rein- 
forcement of a response does not 
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lead to a standard amount of decre- 
ment, but the effect of non-reinforce- 
ment varies with arrangements such 
as those called periodic recondition- 
ing and conditioning at a fixed ratio 
(Chapter 6, Figures 11-13). 

The evidence for spontaneous re- 
covery on which some of the argu- 
ment in the text rests may be found: 
for reflexes, in Humphrey (1933), 
Prosser and Hunter (1936); for con- 
ditioning, in Ellson (1939a), Hilgard 
and D. G. Marquis (1935), Hilgard 
and A. A. Campbell (1936), Hov- 
land (1937c), Porter (1938d), Raz- 
ran (1939h), Rodnick (1937a), 
Switzer (1933). Disinhibition is 
closely related. Experimental evi- 
dence may be found in Hovland 
(1937c), Hunter (1935c, 1938), 
Razran (1939g), Rexroad (1937), 
Reynolds (1936, 1939), Switzer 
(1933), Wenger (1936a). The fail- 
ure reported by Skinner (1936) is 
in part a function of his particular 
theoretical interpretations. 

Evidence on the rate of elicita- 
tion in relation to speed of extinc- 
tion is not conclusive. More rapıd 
adaptation with massing has been 
shown: for reflexes, by Humphrey 
(1933), Prosser and Hunter (1936), 
Hunter (1937); for conditioning, by 
Calvin (1939), Hilgard and D. G. 
Marquis (1935), Mikhailof (1922). 
This is also true for voluntary re- 
sponses (Robinson, 1934). Guthrie 
(1934) has shown that Pavlov’s 
evidence does not support his con- 
tention that massing produces more 
rapid extinction, if reduction is ex- 
pressed per non-reinforced trial. 
Porter found no differences between 
massed and distributed extinction, 
either in conditioned eyelid reac- 
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tions with man (1939), or in the 
extinction of maze-running by rats 
(1938c). 

Specimen studies of the effects of 
drugs on conditioning and extinc- 
tion are: alcohol by Andreyev 
(1934b), Gantt (1935), N. E. 
Miller and Miles (1936); caffeine 
by N. E. Miller and Miles (1935), 
Skinner and Heron (1937), Switzer 
(1935a, 1935b), Wentink (1938), 
Wolff and Gantt (1935); benze- 
drine by C. W. Brown and Searle 
(1938), Searle and C. W. Brown 
(1938), Skinner and Heron (1937), 
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Wentink (1938); bromides and 
other sedatives by Bam (1937), S. 
L. Levin (1935), Maiorov (1933), 
Petrova (1934-1937), Wolff and 
Gantt (1935). 

The instances of failure of extinc- 
tion (or of great difficulty in extinc- 
tion) are found in: Baker (1938), 
Baschmurin and Mulberg (1928), 
Harlow (1937), Harlow and Stag- 
ner (1933), Hilgard and D. G. 
Marquis (1936), Hudgins (1933), 
Kriazhev (1929a), Menzies (1937), 
Osipova (1927), Razran (1935). 


Chapter 6 
STRENGTH OF CONDITIONING 


XPERIMENTATION has reached the stage in which it adds 
E little to report that a given organism can form condi- 
tioned responses, or that a given response can be conditioned. 
The circumstances which modify the course of conditioning, 
extinction, discrimination, become increasingly important after 
the demonstration has been made that such modifications oc- 
cur. If these circumstances could be stated with precision, we 
would have experimental ‘laws’ of conditioning. That is, if 
we could write a formula for the change in rate of condition- 
ing with increasing time interval between conditioned and 
unconditioned stimuli, or with changes in intensity of stimuli, 
then we should have laws comparable, say, to Boyle’s law in 
physics which relates the volume of a gas to changes in pres- 
sure. There are at present no precise quantitative laws of 
conditioning. In this chapter the attempt is made to collect 
specimens of the relationships between independently defined 
variables which may eventually lead to laws. The service- 
ability of the conditioning experiment to learning theory de- 
pends upon the consistency with which such relationships may 
be demonstrated experimentally. 


Strength of Response and Strength 
of Conditioning 


In order to make a quantitative demonstration of the ef- 
fect of independent variables upon the strength of condition- 
135 
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ing, it is necessary to have measures of this strength. In prac- 
tice, the determination of the degree of conditioning depends 
first upon satisfactory measures of strength of response (such 
as drops of saliva, or extent of movement) and second upon 
an inference from the measured vigor of response to the 
strength of conditioning. Not all measures of response are 
suitable as indexes of strength of conditioning. Thus, for or- 
dinary reflexes, reduction of latency accompanies increased 
vigor of movement, so that change in latency may serve as 
one measure of strength of response. In the case of delayed 
conditioned responses, however, the better established delayed 
response has a longer latency than the less well-established 
response. In this case the inference from a given latency 
change to strength of delayed conditioning must be made op- 
posite to the inference in the case of a simple reflex. In situa- 
tions in which there is a high correlation between responses 
during conditioning and the responses during extinction, the 
latter might be used as a measure of the former (cf. Table 4, 
Chapter 5). When this correlation is reduced as it is in some 
experimental situations, resistance to extinction is not a valid 
measure of strength of conditioning. The same sorts of re- 
Sponses are measured in all these cases, but the inferences to 
be made from the obtained measurements depend upon an 
understanding of the total situation. In many cases the per- 
formances provide a direct index to the strength of condition- 
ing, and strength of response may be used as a criterion of 
strength of conditioning, but the exceptions must not be ig- 
nored. 

Among the measures of strength of response used in in- 
ferring strength of conditioning may be mentioned magnitude, 
latency, percentage frequency, total number of responses, and 
rate of responding. Magnitude and latency are descriptive of 
single responses. The other measures are all relative to the 
Opportunities given to respond, i.e., length of time or number 
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of presentations of the conditioned stimulus. Magnitude may 
be measured in various ways: drops of saliva, millimeters of 
pupillary constriction, total oscillation of inspiration and ex- 
piration, tension exerted by a withdrawal movement, dura- 
tion of response, and so on. Latency is measured from the on- 
set of the conditioned stimulus to the beginning of response. 
Percentage frequency, total number, and rate, are obtained by 
counting the responses which meet the specifications for an 
acceptable conditioned response in the given experimental sit- 
uation. Percentage frequency describes the number of re- 
sponses yielded in relation to the number of times the con- 
ditioned stimulus was presented. It is not uncommon during 
extinction to continue presentations until no more responses 
occur. It may then be inferred that the total number of re- 
sponses during extinction bears a relationship to the strength 
of conditioning before extinction. In the arrangement of Skin- 
ner’s (1938) experiment, the number of responses in a given 
time, i.e., rate of responding, is a useful measure, because the 
animal is free to respond again as soon as the first response is 
over. Skinner distinguishes between momentary rate and to- 
tal amount of potential activity, the latter constituting the 
“reflex reserve.” 

If all of these measures are possible, which of them really 
measures strength of conditioning? Such a question is unan- 
swerable in general terms. Practically, any reliable measure 
may prove to have significance in specific comparisons, and 
only the use of proper controls and the study of intercorrela- 
tions between the measures within a given experiment can 
determine which measures are most serviceable. Exact values 
of reported reliability coefficients have little meaning, because 
such coefficients are relative to the method used in fractionat- 
ing the data, to the heterogeneity of the population, and so 
on. With a given population and a given method of comput- 
ing the reliabilities, relative values are of interest. In reports 
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of split-half reliabilities of conditioned flexion responses in 
dogs, conditioned eyelid reactions in man, and conditioned 
knee jerks in man, percentage frequency of response and 
amplitude have been found to have appreciably higher relia- 
bilities than latency (Kellogg and E. L. Walker, 1938b; A. A. 
Campbell and Hilgard, 1936; A. A. Campbell, 1938). 


TABLE 5 


INTERCORRELATIONS OF Various Measures OF CONDITIONED RESPONSES 


Values for conditioned flexion responses are means of rank order correla- 
tions from several sessions with 3 dogs (Kellogg and E. L. Walker, 1938b). 
Values for conditioned eyelid reactions in man are product-moment coeffi- 
cients based on 100 trials with 63 college students (A. A. Campbell and 
Hilgard, 1936). The coefficients for conditioned knee jerks are product- 


moment correlations based on 100 trials with 49 college students (A. A. 
Campbell, 1938). 





Correlation between 
FREQUENCY AND LATENCY AND LATENCY AND 





AMPLITUDE AMPLITUDE FREQUENCY 
Flexion (oR cee Wile Sea + .94 — .22 —.18 
Eyelid (man) TOOTS CoO Ree + .63 —.15 — 54 
Kaes jode (man) eee eeee nee + .63 — .27 — .27 


a a 


In Table 5 are given raw correlations between percentage 
frequency, amplitude, and latency of conditioned responses in 
the three studies cited. The correlation between frequency 
and amplitude is sufficiently high that it may be assumed that 
they measure something in common. The correlation between 
either of these measures and latency is too low to be of sig- 
nificance in prediction. 

F Once measures from which to infer the strength of condi- 
tioning have been selected, it is possible to study the rela- 
tionship of strength of conditioning to various changes in the 
experimental situation. From among the welter of relation- 
ships which have been studied, three classes of experimental 
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variables have been selected for discussion. These are (1) 
capacity of the organism, (2) intensity and (3) repetition. A 
fourth important variable, time interval between stimuli, is 
discussed in the following chapter. 


Capacity of the Organism 


The conditioning experiment assumes an organism adapted 
to the experimental situation, having the necessary sensori- 
motor equipment and associative capacities to yield condi- 
tioned responses in the presence of the chosen stimuli. It 
may be inquired to what extent these capacities are depend- 
ent upon membership in a species, on the one hand, and upon 
the stage of individual development on the other. 

Phylogeny. Within the phylogenetic series, or within mam- 
malia alone, comparisons are extremely difficult to make be- 
cause of the differences in sensitivity to particular stimuli, 
differences in response equipment, and variations in motiva- 
tion. It has been shown, for example, that leg flexion responses 
cannot be conditioned in the opossum, presumably because 
such responses are not part of its natural defensive equip- 
ment (W. T. James, 1937). Limited comparisons may be made 
between species whose nervous systems, sensory equipment, 
and effector mechanisms permit experimentation under cir- 
cumstances which are highly similar. 

Such comparisons have been made by Liddell, W. T. James 
and O. D. Anderson (1934) on the pig, dog, sheep, goat, and 
rabbit. Even with animals as similar as these, differences in 
experimental arrangements are necessary. The sheep, for ex- 
ample, is not a satisfactory subject unless another sheep is 
tethered in the room with it. In general, it was found in ex- 
periments with a tone followed by shock to the foreleg that 
the pig conditioned most rapidly, requiring only a single re- 
inforced trial, while the dog required 3 to 5 trials, the sheep 
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and goat 7 to 9, and the rabbit 14 to 24. Comparison of con- 
ditioned eyelid responses in dog, monkey, and man has been 
made under similar circumstances of experimentation (Figure 
2, p. 32), and the white rat has been added to this series 
by Hughes and Schlosberg (1938). While many common prin- 
ciples of conditioning are applicable to the four species, the 
differences do not correspond in any direct manner to their 
evolutionary levels. 

Ontogeny. The earliest reported conditioning is that of 
the fetus prior to birth. Conditioning has been reported for 
the chick before emerging from the shell (Gos, 1935). The 
human fetus has been conditioned in utero during the last two 
gestation months (Spelt, 1938). Conditioned feeding responses 
have been established by the fifth post-natal day (D. P. Mar- 
quis, 1931). Children between the ages of 44 and 117 days 
have acquired feeding responses in 16 to 72 trials, with no 
consistent change in the rate of acquisition with age (Kantrow, 
1937). Age does appear to make some difference in the ease 
with which discriminatory reactions may be conditioned (Fig- 
urin and Denisova, 1929), but it is not clear whether this is 
a question of the maturation of sensory functions or of the 
capacity to learn the motor differentiation. Mateer (1918) 
found increasing success in conditioning food responses as 
ages increased from within the first two years to within years 
4 and 5. Osipova (1926a), who conditioned finger withdrawal 
to shock, found conditioning progressively more difficult as 
ages increased from 7 to 19. The possibility is great that age 
differences will be found specific to the responses which are 
conditioned. Changes which take place as language develops 
will probably affect some varieties of conditioning more than 
others. With more highly developed symbolic capacities, trans- 
fer should be greater and voluntary factors may become more 
prominent. These are matters urgently requiring more experi- 
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mentation, although the difficulties in the way of crucial ex- 
periments are very great. 

Individual Differences. The ease with which a partic- 
ular organism forms conditioned responses is not a matter 
solely of membership in a species (phylogenetic differences) 
or of age (ontogenetic differences). With comparable stimuli, 
some organisms of the same species and of the same age form 
responses more readily than others. So striking are these dif- 
ferences in dogs that Pavlov was led to a theory of types of 
nervous system. The evidence with respect to individual dif- 
ferences is discussed in Chapter 12. 

In addition to the more enduring differences, there are a 
number of temporary states, such as hunger and satiation, 
health, hormone and drug influences, which affect condition- 
ing as well as other performances of the organism. 


Intensity 


Intensity of Conditioned Stimulus. The relationship 
between the intensity of the conditioned stimulus and the 
strength of conditioning 1s complicated by the effect upon 
response strength. Moderate increases in intensity of condi- 
tioned stimuli tend to increase the magnitude of conditioned 
responses. This effect is similar to the dependence of reflex 
amplitude or of voluntary reaction time upon stimulus in- 
tensity. It has been shown for example by Kupalov and Gantt 
(1927) that the magnitude of the conditioned salivary re- 
sponse in dogs varied with the intensity of the conditioned 
sound stimulus in the roughly logarithmic relationship char- 
acteristic of other responses. The method of constant stimuli 
used by Newhall and R. R. Sears (1933) in determining visual 
thresholds by a conditioning procedure showed that response 
frequency was greater to brighter intensities of lights near 
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the threshold. In these studies, the different intensities of 
conditioned stimuli were tested after the conditioned response 
had been well established. They demonstrate that response 
strength is a function of stimulus intensity, but say nothing 
about the effect on rate or degree of conditioning. 

In order to determine the influence of stimulus intensity 
on strength of conditioning it is necessary to compare groups 
of subjects which have been trained with different intensities 
of conditioned stimulus. This comparison cannot be made 
simply between the final magnitude of the conditioned re- 
sponses, but must be made on tests in which the intensity of 
the conditioned stimulus is equated for the different groups. 
Only in this way can the effect on response strength be con- 
trolled to demonstrate the effect on strength of conditioning. 
Data fulfilling these conditions are found in three studies by 
Hovland on generalization (1937b, 1937c, 1937d). He estab- 
lished conditioned galvanic skin responses in two matched 
groups of human subjects, one with a weak conditioned stimu- 
lus (tone, vibrator), the other with a strong conditioned stim- 
ulus. After training, both groups were tested with both the 
weak and the strong stimuli (and in one study also with in- 
termediate intensities), Computation of the original data pre- 
sented by Hovland demonstrates that there was no reliable 
superiority of one group over the other. At the present time 
there is no evidence that strength of conditioning, distinguished 
from response strength, is a function of the intensity of the 
conditioned stimulus. 

There are undoubtedly limiting intensities for the condi- 
tioned stimulus, so that too strong a stimulus may actually 
reduce the amount of conditioning through interference by 
the original responses which it evokes. Baker (1938), working 
with barely perceptible auditory stimuli, found conditioning 
to be much easier near the threshold than slightly above it. 
(The sounds described by him as subliminal were actually at 
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or above the threshold, as measured by the more usual psy- 
chophysical methods.) Why these barely audible sounds 
should have been more effective than louder ones is at pres- 
ent obscure, but Baker’s results show that the relation of 
intensity to ease of conditioning is not a simple one. If his 
results are substantiated, a ‘too strong’ stimulus may be a 
stimulus far weaker than usually suggested. 

Strength of Reinforcement. The relationship between 
the strength of conditioning and the strength of reinforce- 
ment must be stated differently for each of the varieties of 
reinforcement. 

In conditioning with homogeneous reinforcement, the mag- 
nitude of the unconditioned response should be a direct deter- 
miner of the strength of the conditioned response which de- 
velops. The amplitude of conditioned eyelid reactions of a 
group of college students correlated + .55 + .o6 with the ampli- 
tude of unconditioned reflexes (A. A. Campbell and Hilgard, 
1936). Between the amplitude of conditioned responses and 
knee jerks during conditioning, Schlosberg (1932) found a 
correlation of +.37+.13 and A. A. Campbell (1938) a cor- 
relation of + .26+.09. These correlations, while barely signifi- 
cant, show that the magnitude of the unconditioned response 
is one of the factors responsible for the course of condition- 
ing. Anything which affects the unconditioned response will 
presumably have some effect on conditioning. The most ob- 
vious of these factors is the intensity of the unconditioned 
stimulus. Others, such as motivation and set, through altering 
sensitivity to the unconditioned stimulus, will affect the size 
of response. The effective intensity of the stimulus is ob- 
viously dependent on the state of the organism. 

In conditioning with heterogeneous reinforcement, the effect 
of any one reward may be expected to bear a relationship to 
the magnitude of the relevant responses to that reward. Thus 
a food for which there is appetite should constitute a better 
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reinforcing agent than a food which is disliked or merely tol- 
erated. The strength of reinforcement is related to drive, and 
to the quantity and quality of reward. These factors are meas- 
ured in such a variety of ways that any general statement 
concerning strength of reinforcement is at present impossible. 
Probably the strength can best be described in some defining 
situation, other than the conditioning one, so that it may enter 
the conditioning situation as an independent variable. Too 
intense a drive may weaken reinforcement, possibly through 
the introduction of irrelevant and disrupting emotional states. 

To measure the intensity of an expectation in a situation 
involving derived reinforcement is even more difficult. The 
effectiveness of a secondary reward presumably depends upon 
two factors: the degree to which it has come to signify reward, 
and the effectiveness of the original reward. In the avoidance 
situation the strength of the expectancy may be supposed to 
depend upon analogous factors: first, the degree to which the 
conditioned stimulus has come through repetition to signify 
the impending painful stimulus, and second, the intensity of 
the painful stimulus itself. 

Intensity Relations of Conditioned and Uncondi- 
tioned Stimuli. It is sometimes assumed that unconditioned 
responses must be facilitated by the conditioned stimulus in 
order for conditioning to occur. Some evidence points in this 
direction, for example, the correlation of +.60+.10 which 
Schlosberg (1932) found between the success of conditioning 
and the amount by which the conditioned stimulus facilitated 
the knee jerk to the unconditioned stimulus. This is by no 
means a necessary relationship for conditioning, however, 
since it has been shown at the favorable intervals for eyelid 
conditioning that the conditioned stimulus definitely depresses 
the responses to the unconditioned stimulus (Hilgard, 1933b)- 

Razran (1935) has suggested a principle of favorable ratios 
of excitation. He supposes that there may be a most favorable 
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relationship between the effective intensities of conditioned 
and unconditioned stimuli for conditioning to proceed. While 
many observations pointing toward this interpretation are 
available, the systematic experiments which would be neces- 
sary to formulate with precision the intensity interrelation- 
ships have not been reported. 


Repetition 


Practice Curves. Although there is a current tendency 
among psychologists to subordinate the rôle of repetition or 
exercise in the formation of habits, all agree that repetition 
permits other factors to reveal their effects. 

Numerous practice curves of conditioning show similar 
form because of the uniformities that exist among most con- 
ditioning experiments. Since conditioned responses are usu- 
ally not yielded until several reinforcements have occurred, 
the first portion of the curve is flat, rising from the base as 
conditioned responses appear. A rising inflection of this kind 
is known among psychologists as a positively accelerated 
curve, although it may be described simply as an accelerated 
one, since there are increasing gains with practice. The phase 
of increasing gains tapers off as a maximum is approached. 
This phase, commonly known as negative acceleration, may 
be described as decelerated, since gains decrease as practice 
proceeds. Most conditioning curves show double inflection, 
being at first accelerated, later decelerated. Within the family 
of curves so described, great differences occur, from the con- 
ditioning which is established in a single trial to failure to 
condition in hundreds of trials. Specimen curves may be 
found in Figure 5 (p. 59), Figure 7 (p. 112), Figures 10-13 
(pp. 149-152), Figure 21 (p. 181), Figure 24 (p. 186), Figure 
35 (p. 264). These illustrate the uses made of performance 
curves in the analysis of the influence of different variables. 
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Too many consecutive reinforcements may result in a de- 
crease in the magnitude or frequency of conditioned responses. 
This decremental factor, which Hovland (1936) has termed 
‘inhibition of reinforcement,’ is probably an illustration of 
adaptation (see p. 124f.), since it is produced by massing 
of trials, or continuing reinforcement too long, and shows 
spontaneous recovery during short intervals of time. 

The learning curve for instrumental learning depends in 
part on the degree of environmental simplification achieved 
by the experimenter. Skinner (1938) finds that a single rein- 
forcement commonly initiates a series of lever-pressings in 
his rats at their optimal rate. That is, momentary response 
strength is complete in one trial. Further reinforcements build 
up a ‘reflex reserve,’ best described in conventional condi- 
tioning terminology as resistance to extinction. S. B. Williams 
(1938) has demonstrated that such resistance is proportional 
to the number of reinforcements. Comparisons are difficult 
when both mechanical arrangements and units of measure- 
ment are so unlike that different operations are involved in 
determining changes in strength of response. Youtz (1938a) 
has plotted data obtained by Skinner’s method in somewhat 
more conventional units. He finds that the time required for 
successive IO responses in the lever situation gradually de- 
creases, in a curve roughly resembling that of Thorndike’s 
time-reduction curves for trial-and-error learning. The curve 
cannot be carried beyond the first 30 or 40 responses in this 
manner, because the ingestion of food changes the drive, and 
the ‘learning curve’ becomes more nearly a ‘satiation curve. 

Under some circumstances, the performance curve for 
avoidance learning is comparable to those of classical condi- 
tioning and of rewarded learning (cf. Figure 5, p. 59). When 
voluntary factors enter, responses are often at a maximum 


early, and then tend to drop out (Yacorzynski and Guthrie, 
1937; D. G. Marquis and Porter, 1939). 
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Curves of Extinction. Curves of extinction with re- 
peated non-reinforcement tend to be decelerated, response re- 
duction being more rapid at first, then gradually tapering off 
as a baseline is approached. Some curves show an initial rise 
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Extinction of galvanic skin re- Extinction of galvanic skin re- 
sponse immediately following 24 dis- sponse immediately following 24 
tributed reinforcements. Mean, 20 massed reinforcements. Mean 20 
adult human subjects. (Hovland, subjects. Note the initial rise. If 
1936.) reinforcements were fewer or if time 

elapsed before extinction began, this 
initial rise was lacking. (Hovland, 


1936.) 
before the decrease. The sudden change in stimulating con- 
ditions occasioned by the omitted reinforcement may be con- 
sidered to produce a ‘disinhibition’ following ‘inhibition-of- 
reinforcement.’ Adopting this conjecture, Hovland (1936) 
set up an experiment in such a way that both a decelerated 
curve and one with the initial rise should have been obtained. 
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He predicted that the more usual curve would be found with 
spaced practice, the rising type with massed practice. His re- 
sults, as shown in Figures 8 and 9, agreed with his predic- 
tions. 

Extinction may also be demonstrated for instrumental re- 
sponses which are followed by non-reward. Youtz (1938a) 
has presented curves of extinction for the Skinner situation. 
If responses per tenth of extinction time are plotted, extinc- 
tion time being measured to a criterion of no responses in 20 
minutes, a decelerated curve is obtained, similar to that for 
extinction following conventional conditioning. 

The Distribution of Practice. The decremental influ- 
ences within a series of massed trials disappear with time, so 
that at the beginning of practice after an interval there may 
appear concealed or latent conditioning (H. E. Jones, 1930). 
Spaced practice favors conditioning over massed practice. 
Pavlov took great care not to crowd practice, particularly 
when difficult conditioned responses were being formed. Some 
stimuli were presented never more than once a day. Schlos-. 
berg (1934) found it more effective to use 25 rather than 
200 paired stimulations per day in conditioning breathing 
reactions in rats based on a shock to the tail. Humphreys 
(1940a) has shown that in eyelid conditioning in man the 
end result after two one-hour periods of conditioning on two 
days is very nearly the same whether there are 25 trials per 
period or 50 trials per period. Calvin (1939) has shown con- 
ditioned eyelid responses in man, based on shock, to be 
formed much more readily when trials are spaced at 20 sec- 
onds than at shorter intervals. His r 
ically in Figure 10. 

The effect of spacing on the rate of extinction has been dis- 
cussed in Chapter 5 (p. T39). 

Partial Reinforcement. Not to be confused with the 
distribution of practice, in which inter-trial intervals are 


esults are presented graph- 
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lengthened, is the distribution of reinforcements randomly 
over a fraction of the trials, so that reinforcement is fre- 
quently omitted during the course of training. 

In a preliminary experiment, Pavlov (1927, 384-386) 
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EFFECT or DISTRIBUTION OF PRACTICE ON THE ACQUISITION OF CONDITIONED 
EYELID Response 


The points represent mean values for conditioned responses from three 
groups of human subjects, each subject receiving 50 reinforcements at the 
stated rate. Conditioned stimulus, light; unconditioned stimulus, shock. 
The interval between the light and shock was constant at 400 msec. for all 
subjects. After Calvin (1939). 


found that conditioned responses developed rapidly if rein- 
forcement was given only every second or third trial, but 
failed to develop if reinforcement was given only every fourth 
trial. Similar experiments have been carried out more fully 
by Brogden (19392), Cole (1939), Humphreys (1939a, 
1939c), and Skinner (1938). Brogden found that with dogs 
the flexion response to shock was conditioned as readily with 
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reinforcement 40 percent of the time as with reinforcement 
100 percent of the time, and there was only slight loss with but 
20 percent reinforcement. With food reinforcement, there was 
a slight decrease in conditioning as the proportion of rein- 
forcement decreased, but even with reinforcement only one- 
fifth of the time, conditioned responses appeared on four-fifths 
of the trials. Humphreys found essentially the same thing for 
conditioned eyelid responses and conditioned galvanic re- 
sponses in man, conditioning being as satisfactory with rein- 
forcement half of the time as with reinforcement every trial. 
One of his rather striking findings was that extinction was 
slower in the group which had received only half the reinforce- 
ments. This was true both for conditioned eyelid responses and 
for conditioned galvanic responses. He has offered evidence for 
expectancy as a partial interpretation of these results, through 
a confirmatory experiment (1939b). He has since repeated the 
experimental findings with the lever-pressing response of rats 
(19400). 

Skinner has reported a number of results for instrumental 
learning under an experimental arrangement which he de- 
scribes as ‘periodic reconditioning’ (Skinner, 1938, 116-166). 
In this arrangement, pressing does not deliver the pellet of 
food regularly, so that most of the trials are not reinforced. 
However, a single reinforcement occurs periodically, for ex- 
ample, every 3 minutes, or every 9 minutes according to the 
plan of the experimenter. Under these circumstances, the rate 
of responding tends to become uniform provided the hunger 
drive remains relatively constant (Figures 11 and 12). Be- 
cause of the regularity of Tesponses, it is possible to express 
the rate of responding as an extinction ratio, that is, as the 
st ropes oper 
sees each Paes For example, under stand- 

> this ratio is approximately 18:1; that 
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FIGURE II 


Responses within one session of 
periodic reinforcement: 


The curves are reproduced from 
the kymographic records of individ- 
ual rats under each of the four condi- 
tions. Reinforcement was given every 
3 minutes, 6 minutes, 9 minutes, and 
12 minutes, respectively. Note that 
the more frequent the reinforcement, 
the more rapid the rate of respond- 
ing, although each rate is relatively 
uniform. (Skinner, 1938, p. 121.) 


FIGURE 12 


Responses within repeated sessions 
of periodic reinforcement: 


Responses of the same rats whose 
records are given in Figure II are 
here accumulated for successive daily 
sessions. The uniformity of rate per- 
Se asain (Skinner, 1938, p. 
120. 
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is, for every reinforcement the rat tends to press the lever 
about 18 times. 

Very different results are found when the final member of 
a counted series of pressings is reinforced. After appropriate 
preliminary training it is possible to secure a high level of 
response by reinforcing only the final one of every 192 re- 
sponses. Between reinforcements, a very rapid series of re- 
sponses occur (Figure 13). Under these circumstances, the 
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ÅCCELERATED RATE or RESPONDING Between REINFORCEMENTS WITH 
REINFORCEMENT AT A Frxep Ratio 

Responses from individual rats reinforced every 48, 96, and 192 responses, 

as indicated by the horizontal lines. Under these circumstances very high 


Tates of responding develop, the highest rate being found with the lowest 
frequency of reinforcement. (Skinner, 1938, p. 288.) 


less frequent the reinforcement, the higher the rate of re- 
sponding. Whether classical conditioning could be established 
by reinforcing only once in 192 trials is extremely doubtful. 
The difficulty in moving directly from classical conditioning 


to instrumental conditioning is indicated by some of these 
striking differences. 


Following periodic reinforcement, the decline in rate of 


response during extinction is much slower than following 
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ordinary every-trial reinforcement (Skinner, 1938, p. 133f.). 
This is in full agreement with the results of Humphreys for 
partial reinforcement. It is impossible to compare the quanti- 
tative changes, however, because of the difference in the kinds 
of measurements involved. 

Repeated Conditioning and Extinction. Following ex- 
tinction, reconditioning is very rapid, even though time for 
spontaneous recovery has not intervened. The precise ex- 
perimental conditions will, of course, affect the quantitative 
relationships, so that ratios of relearning time to original learn- 
ing must be specific, rather than general. As specimen find- 
ings, however, the results of two experiments with dogs may 
be cited. Finch and Culler (1935) report for flexion responses 
a reconditioning after extinction in from 20 to 25 percent of 
the original trials, while Hilgard and D. G. Marquis (1935) 
report for eyelid responses relearning in from 20 to 40 per- 
cent of the original trials. 

Among the factors determining the rate of reconditioning 
following extinction are, first, the extent to which extinction 
was carried, and, second, the number of prior reconditionings 
after extinction. These factors are described in experiments 
by Brogden, Lipman, and Culler (1938). As an illustration 
of the effects of repeated extinction and reconditioning, one 
dog is reported to have required after a first extinction, 18 
reinforcements; after a second, 11; after a third, 5; after a 
fifth, 1; after a seventh, 2. Because the third and fifth extinc- 
tions had included 450 extra non-reinforced trials, the seventh 
850 added trials and yet reconditioning was prompt, extinc- 
tion does not appear to have gone ‘below zero’ in the re- 
peated extinction-reinforcement series. 

Successive extinctions, like successive reconditionings, also 
tend to become more rapid until a single non-reinforcement 
may suffice (Pavlov, 1927, 54f.). Ellson (1940) carried out 
IO successive extinctions, separated by 25 minutes, of the 
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lever pressing response in the Skinner apparatus. The average 
number of responses for I2 rats in the successive sessions was 
46, 16, 6, 6, 5, 3, 2, I, I, o Shorter intervals between extinc- 
tions led to a more rapid disappearance of the response. 


Summary 


A distinction was drawn between strength of response and 
strength of conditioning. Strength of response refers to the 
vigor of performance as measured by such criteria as magni- 
tude or frequency. Strength of conditioning is inferred from 
strength of response, but is not to be identified with it. This 
is obvious, for example, when a conditioned response has been 
reduced to zero amplitude. Strength of conditioning may per- 
haps go ‘below zero,’ even though strength of response can- 
not. This would be inferred from tests of spontaneous recov- 
ery. A facilitated response may be increased in magnitude 
without thereby increasing strength of conditioning. Measure- 
ment is always of strength of response; strength of condition- 
ing is always an inference from measured responses. 

By appropriate control experiments it is possible to explore 
systematically the influence of various experimental variables 
on strength of conditioning. The influence of membership in 
a species, of age, of intensity of stimuli, of repetition of rein- 
forcement and non-reinforcement, were examined in this chap- 
ter. Many consistencies are discovered, but the data are not 


yet sufficiently ordered for quantitative laws of conditioning 
to be stated. 


Notes 


Measures of the Strength of tioned and unconditioned responses, 

Conditioned Responses cf. A. A. Campbell (1938), Hum- 
phreys (1940b), Kellogg and E. L. 
Walker (1938b). Reliabilities can, 
of course, be inferred from other 


For Statements of reliability of 
various measures of both condi- 
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studies in which satisfactory statis- 
tical differences have been secured, 
but they have seldom been reported 
in the form of reliability coefficients. 
A correlation of + .66 (P.E. + .o9) 
between the amplitude of condi- 
tioned knee jerks within training 
trials and within test trials (Schlos- 
berg, 1932) is a sort of reliability 
coefficient. 

If individual measures are reli- 
able, low correlations between them 
may mean that they are measuring 
different processes. Thus Razran 
(1935) believes that incidence (i.e., 
percentage frequency) and magni- 
tude of conditioned responses are 
variables differently related to phys- 
iological and psychological (i.e., 
attitudinal) factors, incidence being 
more subject to attitudinal control 
than magnitude. For intercorrela- 
tions of various measures in addi- 
tion to those described in the text, 
cf. Razran (1935, p. 64). 

Skinner (1938, pp. 15, 27) distin- 
guishes between reflex strength and 
magnitude of response. Magnitude 
is a function of intensity of stimula- 
tion, so that a strong reflex may 
exhibit a response of low magnitude 
if the stimulus is of low intensity. 
He uses magnitude, therefore, as one 
Measure of what we have called 
strength of response. Reflex strength, 
on the other hand, is a momentary 
function of a certain amount of 
available activity which Skinner 
speaks of as the ‘reflex reserve.’ 
What we have called strength of 
conditioning corresponds loosely 
both to his reflex strength and to 
his reflex reserve, but the concepts 
cannot be coordinated strictly. 
Skinner’s concept of the reserve as 
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a measure of strength of condition- 
ing has been adversely criticized by 
Ellson (1939b). 


Capacity of the Organism 


For further details on phyloge- 
netic comparisons, see Razran 
(1933a); for ontogenetic compari- 
sons, Razran (1933b). A highly 
speculative theory may be found 
in Razran (1935). Wenger (1936b) 
found some diffculty in condition- 
ing very young infants. His results 
and those of Kantrow (1937) are 
commented upon by Irwin (1939). 

No attempt has been made in the 
text to treat the details of motiva- 
tion and drive, and other influences 
on the state of the nervous system 
which affect strength of response. 
Some pertinent references are given 
here. 

Motivation has been thoroughly 
reviewed by Young (1936). For dis- 
cussion of hunger and activity, in 
relation to conditioning, see Babskij 
and KEidinova (1933), Elliott and 
Treat (1935), Finch (1936, 1938a), 
Sackett (1939), Skinner (1938), 
Zener and McCurdy (1939). 

The effect of diet upon condi- 
tioned responses has been studied 
by Arkhangelsky (1929), Crisler 
(1936), Frolov (1925b), Frolov 
and Charitonov (1931), Kleitman 
(1927), Machtinger (1933), Vorob- 
yov (1932), Wolowick (1930), 
Yushchenko (1927). The relation of 
conditioning to disease has been dis- 
cussed by Andreyev (1935), Frolov 
and Charitonov (1931), Krasno- 
gorski (1931), and Kriazhev 
(1929b). 


Endocrine influences have been 
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extensively studied, the thyroid by 
Asimov (1928), Crisler, Booher, 
Van Liere, and J. C. Hall (1933), 
Kleitman and Titelbaum (1936), 
Liddell (1926b), Liddell and Bayne 
(1927), Liddell and Simpson 
(1926), Pribytkova (1936), Shastin 
(1930a, 1930b), Zavadovsky et al. 
(1928-1932); the parathyroid by 
Andreyev and Pugsley (1934) and 
Chechulin (1929); the hypophysis 
by Kriazhev (1933); the adr-nal 
by Liddell, O. D. Anderson, Koty- 
uka, and Hartmann (1935) and 
Yurnan (1929); the gonads by 
Arkhangelsky (1931a, 1931b, 1935), 
A. M. Pavlova (1937), Petrova 
(1936, 1937), Vanderplank (1938). 

Conditioned reflexes in relation 
to sleep have been studied by Kleit- 
man (1930), Mishchenko (1935b, 
1936a), and Rojansky (1914); while 
the effects of hypnosis have been in- 
vestigated by Fisher (1932), Erick- 
son (1938a, 1938b), S. L. Levin 
(1934), Lundholm (1928, 1932), 
Mishchenko (1932, 1935, 1935b), 
Nevsky and S. L. Levin (1932), 
and Scott (1930). 

X-ray radiation of the brain has 
been studied with conditioning tech- 
niques by Lyman, Kupalov, and 
Scholz (1933), and Nemenow 
(1934). Increased sensitivity of 
dogs to auditory stimuli following 
radiation of the head was found by 
Girden (1935), and Girden and 
Culler (1933, 1934). Brogden and 
Culler (1937) showed that the effect 
was accomplished by a pituitary in- 
fluence upon blood sugar and a con- 
sequent change in the density of the 
endolymph of the inner ear. 

Persisting individual differences 
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and theories of types are discussed 
in Chapter 12. 


Intensity 


Other evidence on effect of in- 
tensity may be found in Gantt 
(19382), Kupalov, Lyman and Lu- 
kov (1931), Mishchenko (1936b), 
Zevald (1933). 

In comparing the relative impor- 
tance of intensity and time order, 
Stephens (1934a) found time order 
to be the more important variable. 
This need not deny some rôle to 
intensity, however. 

The applicability of a concept 
of dominance has been proposed 
by Razran (1930) and Rexroad 
(1933). A response, in order to 
be conditioned, must be dominant 
over other responses. This is not 
to be confused with the physiolog- 
ical concept of dominance proposed 
by Ukhtomsky (1926, 1938) and 
endorsed in relation to conditioning 
by Bekhterev (1932, p. 256). 


Repetition 


Numerous performance curves 
plotting acquisition and extinction 
are now available. Some specimen 
conditioning curves are to be found 
in Brogden, Lipman and Culler 
(1938), Calvin (1939), Cohen, Hil- 
gard, and Wendt (1933), Finch and 
Culler (1935), Garvey (1933), 
Girden, Mettler, Finch and Culler 
(1936), Hilgard (1931, 1933a), Hil- 
gard and A. A. Campbell (1936, 
1937), Hilgard and D. G. Marquis 
(1935, 1936), Hovland (19374), 
D. P. Marquis (1931), D. G. Mar- 
quis and Hilgard (1936, 1937), 
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Hunter (1936b), Kantrow (1937), 
J. Miller (1939a, 1939b), J. Miller 
and Cole. (1936), Razran (1935), 
Rodnick (1937a), R. R. Sears 
(1934), Wendt (1930). Curves for 
simple instrumental responses may 
be found in Grindley (1932), Skin- 
ner (1932, 1938), Youtz (1938a). 
Culler’s (1928) suggestion that the 
learning curve is of S-shape was 
first used in relation to the per- 
formance curve of conditioning by 
Kleitman and Crisler (1927). The 
double-inflected curve suggests other 
processes which, by analogy, have 
sometimes been supposed to under- 
lie conditioning. Thus the curve of 
autocatalysis is of this sort, and an 
autocatalytic process has been sug- 
gested as the physiological basis for 
learning. The curve of growth is of 
this form; therefore growth may 
underlie conditioning. The integral 
of the probability curve (the ogive) 
is also of this form. Hence condi- 
tioning may consist in the accumu- 
lation of learned elements, the meas- 
ured result being a complex of 
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subordinate conditionings which be- 
haves in some respects like a normal 
population of responses. Individual 
curves depart so widely, however, 
from a true ogive that the param- 
eters must be worked out with 
greater care before such theories can 
be useful. 

The typical decelerated curve of 
extinction is illustrated in Cole 
(1939), Humphreys (1939a), 
Kleitman and Crisler (1927), J. 
Miller and Cole (1936), Switzer 
(1933). Extinction showing an ini- 
tial rise may be found in Hilgard 
and D. G. Marquis (1935), Hov- 
land (1936), Hudgins (1933), 
Scott (1930), Switzer (1930). The 
course of extinction of an instru- 
mental response is shown in Hunter 
(1935d), Youtz (19382). That 
there is no one general curve of 
extinction is emphasized by Brog- 
den and Culler (1935), Finch and 
Culler (1935), and Hunter (19352). 
See also Brogden, Lipman and Cul- 
ler (1938), Humphreys (1939a), 
J. Miller and Cole (1936). 


Chapter 7 
GRADIENTS OF REINFORCEMENT 


HE RATE of conditioning is markedly affected by the 
je temporal relationship of the stimuli and responses in- 
volved. In classical conditioning, the important time interval 
as usually stated is that between conditioned and uncondi- 
tioned stimuli, as indicated in the summary from Pavlov in 
Chapter 2 (p. 44f.). In instrumental conditioning several other 
time relations are involved. In a maze, for example, one may 
consider the time required to run from a given bifurcation to 
the food-box as of some importance in the elimination of the 
blind alley at that choice-point; a delay may be introduced 
between the time of entering the food-box and the delivering 
of the food reward. Such intervals are measured between re- 
sponses, rather than between stimuli, and the variables are 
defined with differences sufficient to make the empirical laws 
of one inapplicable to the other, no matter how similar they 
may be in certain superficial aspects. 

. When intervals are introduced between events in condi- 
tioning experiments, it is commonly found that there is a 
most favorable interval, with longer or shorter intervals be- 
coming progressively less effective. This gradual decrease in 
effectiveness of conditioning may be described as a sort of 
gradient. Since the interval js ordinarily terminated by an 
unconditioned stimulus, or a reward, or a punishment, it is 
ecbVemient io raie to the whole dess o sudh gradients as 
gradients of reinforcement. Within this class there are sub- 


varieties, in which the interval is measured between different 
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features of the experimental situation. Among the several 
gradients of reinforcement may be mentioned (1) the tem- 
poral gradient in classical conditioning, which refers to the 
significance for conditioning of the interval between condi- 
tioned and unconditioned stimuli, (2) the temporal gradient 
in avoidance learning, referring to the interval between the 
conditioned stimulus and the punishment, (3) the gradient 
of reward (temporal aspects), referring to the delay between 
the conditioned instrumental response and the reward, (4) 
the gradient of reward (non-temporal aspects), referring to 
the interval between the conditioned response and the reward 
expressed as distance in space or as remoteness in a series, 
(5) the temporal gradient within delayed reaction experi- 
ments, in which the instrumental response to the conditioned 
stimulus is prevented for different periods of time, although 
the reward appears promptly once the response is performed, 
(6) the goal-gradient, which is a complicated manifestation 
of the results of repeated trials in which a series of acts have 
been reinforced by a common goal situation. Only confusion 
can result from treating these several situations as though 
they were alike, although there are, of course, analogies be- 
tween them. Each of the above-mentioned gradients will be 
considered in the light of present experimental evidence, be- 
fore any harmonization is attempted. 

The Temporal Gradient in Classical Conditioning. 
The simultaneous conditioned response was considered by 
Pavlov to be basic to the others, and its formation usually 
preceded the establishment of other types. The usual proce- 
dure for establishing a delayed conditioned response in Pav- 
lov’s laboratory, for example, was to condition simultaneously 
and then to postpone the unconditioned stimulus five seconds 
each day until the desired delay was reached. Intervals up 
to five seconds were considered simultaneous by Pavlov, so 
that it may be inferred that within this interval the differ- 
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ences in ease of conditioning were not very pronounced. It 
is evident that strict simultaneity is not an essential condi- 
tion, yet near-simultaneity is more favorable than longer in- 
tervals. This suggests some sort of gradient, whereby there 
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FIGURE 14 
TEMPORAL GRADIENT IN CLASSICAL CONDITIONING 


i The effect of interval between conditioned and unconditioned stimuli in 
uman eyelid conditioning. Conditioned stimulus, click in headphones; 


unconditioned stimulus, shock to skin surface above infraorbital nerve. 
(Bernstein, 1934.) 


is a gradual transition from favorable to less favorable inter- 
vals between stimuli. 

Such gradients have been determined for motor conditioned 
responses by Bernstein (1934) for man, and by Kappauf and 
Schlosberg (1937) for rats. Some of their results are charted 
in Figures 14 and 15. The intervals explored are those con- 
sidered almost simultaneous’ by Pavlov, and such continu- 
ous functions may well be described as temporal gradients. 
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Bernstein’s results show that backward conditioning is 
slight. The later results from Pavlov’s laboratory reversed 
the earlier denials of the possibility of backward condition- 
ing, but they also showed that backward conditioned responses 
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FIGURE 15 
TEMPORAL GRADIENT IN CLASSICAL CONDITIONING 


The effect of interval between conditioned and unconditioned stimuli in 
conditioned respiration of the white rat. Conditioned stimulus, buzz; un- 
conditioned stimulus, shock to right foreleg. Frequency of leg flexions was 
low under all conditions, and respiration proved a more satisfactory index 
of conditioning. (Kappauf and Schlosberg, 1937.) 


can be formed only with difficulty and are evanescent when 
once formed. Strict physical simultaneity is likewise an un- 
favorable temporal relationship. Both investigations show 
that it is desirable for the conditioned stimulus to precede 
the unconditioned by intervals above 200 msec. There are a 
number of independent but less complete studies showing 
the possibility and difficulties of backward conditioning, and 
the disadvantage of strict simultaneity as compared with in- 
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tervals at something over one-fifth of a second. The actual 
range of satisfactory time relations, and the slope of the gra- 
dient, probably vary for different responses and for different 
organisms. It is quite possible, for example, that favorable 
intervals for a long-latency response such as the galvanic or 
salivary reaction may differ from the short-latency responses 
charted in Figures 14 and 15. Unfortunately, systematic 
studies of temporal gradients within other examples of classi- 
cal conditioning have not been made. 

The most favorable interval appears to be somewhat greater 
than the latency of the conditioned response, thus permitting 
the conditioned response to anticipate the unconditioned stim- 
ulus. Two facts are important in this connection: (1) when 
the interval between conditioned and unconditioned stimuli 
is longer than the minimum latency of the conditioned re- 
sponse, the response temporally anticipates the uncondi- 
tioned stimulus; (2) intervals too short to permit the antici- 
patory reaction to appear are unfavorable for conditioning. 
‘These empirical findings support the interpretation of the con- 
ditioned response as a readiness or preparatory reaction. 
Whether or not such an interpretation is accepted, the mere 
fact that the conditioned response tends to precede the un- 
conditioned stimulus may be of very great theoretical signifi- 
cance (Chapters 9 and 10). 

In classical conditioning, the interval between stimuli in- 
fluences latency of response in a systematic way. The mini- 
mum latency of conditioned responses in striate muscle ap- 
pears to be that of voluntary reactions (Hilgard, 1931). These 
latencies are normally over 100 msec., longer than the latencies 
of the unconditioned reflexes on which the conditioned re- 
sponses are based. The latency distributions of conditioned 
eyelid responses in rat, dog, and monkey, show very similar 


characteristics, with latencies longer than those of uncondi- 
tioned reflexes. 
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While the minimum latency of conditioned responses is 
determined in part by the sensori-motor systems involved, re- 
sponses formed with increased intervals between the condi- 
tioned and unconditioned stimuli have longer latencies which 
are determined by other considerations. The latent periods of 
trace and delayed conditioned responses are roughly propor- 
tional to the interval of delay. The latency is ordinarily less 
than the interval, so that conditioned responses appear an ap- 
preciable amount of time before the onset of the uncondi- 
tioned stimulus. This fact is evident from many of Pavlov’s 
tables (1927, pp. 91—102). With the conditioned stimulus act- 
ing alone for 30 seconds, a short delay of 3 seconds is reported; 
when the conditioned stimulus acts for 120 seconds the long- 
est delay reported is 75 seconds; when acting for 180 seconds, 
the longest delay is less than 150 seconds. Thus with 3-minute 
delays conditioned salivation begins at least half a minute 
before the food is presented. 
= Latencies of trace and delayed responses have been meas- 
ured in the studies of Switzer (1933) and of Rodnick (1937a) 
on galvanic skin responses in man. With intervals of 16 to 
20 seconds between the conditioned and unconditioned stimuli 
the conditioned response increased from an original mean 
latency of 5 seconds to a final mean latency of 10 seconds, 
whether or not the delay was established directly or following 
simultaneous presentations. An interesting observation made 
by both writers is that the latency of the delayed or trace re- 
sponse tended to decrease overnight to its original lower value, 
so that in a long series of sessions it had to be reestablished 
each day. The delay was, however, established more promptly 
each day, so that the training of the preceding days was not 
actually lost. 

The greater difficulty of establishing trace responses as com- 
pared with delayed responses was noted by both Switzer and 
Rodnick, in agreement with Pavlov. 
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The Temporal Gradient in Avoidance Learning. H. 
M. Wolfle (1930, 1932) has reported experiments on the 
effect of time interval on conditioned finger withdrawal from 
shock. Subjects were instructed to lift their fingers at the 
shock, and it was possible to avoid the shock by lifting the 
fingers in advance of it. Hence the experiment should be classi- 
fied as avoidance learning, involving voluntary factors. Since 
time interval between the conditioned stimulus and the shock 
has meaning only on those trials in which the shock occurs, 
the gradient described is legitimately considered a temporal 
gradient similar to that of classical conditioning, although 
there are, of course, complicating factors. The results of 
Wolfle’s studies are plotted in Figure 16. In common with 
Bernstein’s results for classical conditioning (Figure 14), 
backward conditioning is slight, intervals near simultaneity 
are unfavorable, and intervals above 200 msec. are most 
favorable. That intervals of 1 second or more were found un- 
favorable by Wolfle may be due to the tendency to restrain 
voluntary conditioned responses when it is too evident that 
they are being made (D. G. Marquis and Porter, 1939). 

Other illustrations of avoidance reactions in which the inter- 
val is measured between a conditioned stimulus and the pun- 
ishment may be found in the experiments of Carr and A. S. 
Freeman (1919), Yarborough (1921), and Warner (19328). 
In each of these, rats were warned by a buzzer of a closed 
door or of a shock, and learned to make anticipatory avoid- 
ance reactions. Intervals of 1 second and less between the 
buzzer and the annoyance were found most favorable. Yar- 
borough demonstrated some backward conditioning at I 
second. Warner found some conditioning with forward inter- 
vals as long as 20 seconds, but found 30 seconds definitely too 
long for his rats. 
The Gradient of Reward: Temporal Aspects. In sim- 


ple instrumental learning leading to a reward, the interval 
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to be measured is not that between stimuli, but between the 
instrumental response and the reward. Furthermore, that 
which is strengthened is not a response similar to that at the 
end of the interval (as in classical conditioning), but the instru- 
mental response which is at the beginning of the interval. The 
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Ficure 16 
TEMPORAL GRADIENT IN AvolIDANCE CONDITIONING 


Conditioned finger withdrawals to a sound when the sound was presented 
at different intervals following and preceding a shock to the middle finger. 
(H. M. Wolfle, 1930, 1932.) 


influence on conditioning of the interval between a response 
and its reward may be called the gradient of reward, as sug- 
gested in Chapter 4 (p. 84). 

In some preliminary experiments, Skinner (1938, pp. 72ff.) 
found delays of 1, 2, 3, and 4 seconds equally efficacious in the 
lever-pressing situation. The delay was introduced between 
the pressing of the lever and the delivery of the food. While 
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a temporal gradient appeared to be lacking here, Skinner 
found such a gradient in other forms of his experiment 
(139ff.). Following periodic reconditioning (in which rein- 
forcement occurred every 4 or 5 minutes) delays of 2, 4, 6, 
and 8 seconds were introduced before the periodic reinforce- 
ments. The intervals progressively decreased the rate of 
responding; the rate declined about one-third through the 
introduction of the 2-second postponement of reward. Skinner 
interprets the differences as a reduction in the reserve (i.e., as 
reduced resistance to extinction) rather than as retarded 
acquisition, and hence distinguishes between his investigation 
and those of delayed reward, next to be described. 

In the more usual instrumental learning situation, in which 
a sequence of acts leads to a reward, postponing the reward 
after the terminal act retards learning. There have been a 
number of experiments on delayed reward, of which Wolfe’s 
(1934) may be taken as representative. He found that the 
learning by rats of a single T-maze was much slower even 
with delays as short as 5 seconds. Longer delays, up to 20 
minutes, produced progressive decreases in the efficiency with 
which the task was mastered. A visual discrimination habit 
was only slightly affected by a 30-second delay, although this 
as well as longer delays showed deleterious effects. In Wolfe’s 
study, the greater part of the effectiveness of the reward was 
lost in the first minute. 

These studies of delayed reward have in common a rather 
steep gradient, whereby delays in reinforcement of a few 
seconds greatly affect the learning. While the evidence is in- 
conclusive, the gradient appears to be somewhat less steep for 
rewarded learning than for classical conditioning. There is a 
complication, however, in that the effect situation is always 
analyzable into a series of acts, which may be discrete stimulus- 
hence Sequences or an integrated series of performances 
eading to a goal. Order of events may in such cases be more 
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important than time interval, and the gradient of reward may 
be expressed in spatial and sequential terms as well as in 
temporal ones. 
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FIGURE 17 
Tue SPREAD oF EFFECT, A FORM OF THE GRADIENT OF REWARD 


The figure summarizes several studies of repetitions of word-number pairs, 
when most of the pairs are followed by ‘punishment’ (experimenter says 
“Wrong”), others by ‘reward’ (experimenter says “Right”). Each sequence 
required about 3 seconds, so that the nine steps illustrated represent a range 
of about 27 seconds. The degree of remoteness is probably a function of the 
number of steps rather than of the precise intervals of time. After Thorn- 


dike (1935, p. 34). 


The Gradient of Reward: Non-temporal Aspects. 
Thorndike’s studies of the spread of effect illustrate non- 
temporal aspects of the gradient of reward. One of his gradi- 
ents is reproduced in Figure 17. The subject was given a word 
and told to say any number from 1 to 6. Following his reply 
the experimenter said either “Right” or “Wrong.” On further 
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repetitions, the effect of the previous announcement of “Right” 
or “Wrong” resulted in excesses over chance as indicated in 
the figure. The rewarded connection (“Right”) was repeated 
much beyond chance expectancy; neighboring connections 
(“Wrong”) were repeated, in spite of being ‘punished,’ be- 
cause of their proximity to the reward. It is to be noted that 
the gradient is expressed in terms of steps. The interval be- 
tween steps was about 3 seconds in the experiments from 
which Figure 17 was constructed; but the form of the gradient 
is little affected if the interval is as short as 1.5 seconds be- 
tween steps. This is not, then, strictly speaking, a temporal 
gradient. It is a gradient expressed in units of successive stim- 
uli presented to the subject, and may be compared with a 
similar procedure used with animals by Hunter (1936a). 

The bi-directional gradient for maze learning reported by 
Muenzinger, Dove, and Bernstone (1937) is based upon errors 
in successive units of an endless maze in which the animals 
run from goal to goal. Like Thorndike’s spread of effect the 
results are given for successive response units, and it is not 
clear whether the function is one of (a) the successive order 
of choices, relatively independent of distance or time interval, 
(b) spatial separation of choice points, or (c) temporal sepa- 
ration of responses. The results agree with Thorndike’s spread 
of effect, in that responses are strengthened on both sides of 
the goal; that is, both prior to and following reinforcement. 

In view of the lack of definitive experiments, the concept 
of the gradient of reward may be retained as a general one to 
describe the strengthening of an act by neighboring reinforce- 
Ment in proportion to remoteness, however that remoteness is 
defined. The more detailed statement of the gradient as a 
function of time, distance, and order of events, awaits further 
evidence. 


The Temporal Gradient within Delayed Reaction Ex- 
Periments. In order to describe the delayed reaction ex- 
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periment in terms of conditioning, it is necessary to consider 
the lifting of the cup which conceals the food, or the opening 
of the box which contains it, as an instrumental response re- 
inforced by the food reward. The experiment consists in show- 
ing the animal the cup or the box as it is being baited, so that 
there is an immediate tendency to give the conditioned re- 
sponse. This tendency to respond is blocked mechanically by 
screening the food device or by removing the animal. Thus 
a delay is introduced between the presentation of the con- 
ditioned stimulus and the possibility of responding to it. The 
longer this delay, the less accurate the choice becomes. This 
gradient of success with delay is not ordinarily considered a 
problem within the sphere of conditioning, but it is described 
at this point as one among many arrangements for interrupt- 
ing a learned act by introducing a temporal interval before the 
act may be performed. The delayed reaction experiment is 
discussed in another context in Chapter 10. 

The Goal Gradient. The gradient of running speed for 
the rat in a straight alley, as reported by Hull (1934b), is a 
spatial gradient, expressed in units of distance from the goal. 
If running through the different segments of the path is con- 
sidered a series of discrete responses, each reinforced by the 
common reward at the end of the maze, it may be predicted 
from the gradient of reward that in a given number of repe- 
titions those responses nearer the goal will be more strongly 
conditioned than those more remote. If running speed corre- 
sponds to strength of conditioning, running speed should be 
faster as the goal is neared. Actually, an integral performance 
of this sort cannot be completely predicted in terms of the 
conditioning of responses according to their distance from the 
goal. One complication which Hull found was a definite retar- 
dation in that section just preceding the goal, representing a 
much more pronounced slowing up than that required to pre- 
vent the rat’s bumping the end of the alley (cf. Figure 27, 


170 CONDITIONING AND LEARNING 


p. 213). It should be noted that the goal gradient as meas- 
ured in an alley maze may be predicted in part from the 
gradient of reward, but is not itself a direct measure of an 
uncomplicated gradient of reward. The spatial gradient for 
avoidance responses, reported by Bugelski and N. E. Miller 
(1938), is similarly related to the temporal gradient in avoid- 
ance learning. Like Hull’s goal gradient, the behavior is 
measured in an alley situation in which a common punishment 
affects the behavior along the path in the approach to it. 

The temporal gradient reported by J- S. Brown (1939) is re- 
lated both to the temporal gradient of avoidance and to the goal 
gradient. The problem was not to find the most favorable 
interval for the learning of avoidance behavior, but to find to 
what extent avoidance behavior was manifested at different 
intervals from a time of usual punishment. Shock stimuli were 
presented at standard intervals of 12 seconds. The tendency to 
give avoidance responses was tested by presenting a less in- 
tense shock at different times within the 12-second interval. A 
gradient was shown, more intense reactions being given near 
the time of usual shock. 

These gradients of running speed and of avoidance have in 
common a single reinforcing state of affairs affecting behavior 
remote from it in time or space. The gradient as tested is a 
performance gradient, following learning, and is in some sense 
the result of learning in accordance with the other gradients. 
A gradient which refers to the relative efficacy of different 
intervals between single responses and their rewards may not 
apply to the strength of individual members of a series with 
a common reward. Only by studying the goal gradient is it 
Possible to determine whether or not the gradient determined 
for single responses is also applicable to the case of a series of 


acts with a single reward. Similar considerations apply with 
respect to gradients of avoidance. 
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The Problem of Contiguity 


To the extent that the gradients which have been described 
represent associative strength (i.e., strength of conditioning) 
as a function of nearness in time and space, they may be con- 
sidered empirical forms of the law of contiguity (Robinson, 
1932, p. 76). It is possible to leave the relationships in this 
empirical form, or to attempt to account for the gradients in 
terms of some more fundamental conception of contiguity. 

Because of an abhorrence of ‘action at a distance’ many 
writers prefer to infer that the events actually associated in 
conditioning are genuinely simultaneous, that is, contiguous 
in the sense of being strictly side-by-side or overlapping. When 
events apparently separated in space or time are associated, 
these writers look for processes which bridge the gap between 
the items. The most common form of this conception is that 
of a stimulus-trace which persists until it is contiguous with 
the response conditioned. Pavlov endorsed this concept in his 
notion of the trace conditioned reflex, in which the conditioned 
stimulus ceased to act before the unconditioned stimulus was 
presented. Pavlov assumed that some trace of the stimulus 
persisted in the organism. 

A related point of view has been espoused by Guthrie 
(1933). According to him, the inferred ‘true’ conditioned 
stimulus is usually produced by the subject’s response to the 
presented stimulus. This ‘true’ conditioned stimulus is said 
by him to be always coincident with the unconditioned re- 
sponse. Since the ‘true’ conditioned stimulus depends for its 
arousal upon the presented conditioned stimulus, it follows 
that the most favorable interval for securing simultaneity of 
‘true’ conditioned stimulus and reinforcing response will be 
one in which there is a short forward order between the pre- 
sented conditioned stimulus and the unconditioned stimulus. 
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According to this interpretation, backward conditioning will 
be possible only if the unconditioned response persists long 
enough for it to be coincident with the ‘true’ conditioned 
stimulus. Since the arrangement of backward conditioning is 
unfavorable for such coincidence, it follows that backward 
conditioning should be relatively unsuccessful. These con- 
jectures have never been directly verified. 

Hull (1937) has also been making use of the conception of a 
stimulus-trace, expressed as a diminishing ‘neural reverber- 
ation.’ His conception of contiguous conditioning is somewhat 
more complicated than Guthrie’s. Like Guthrie, he believes 
that conditioning takes place only between a stimulus-trace 
and a response contiguous with it, but, unlike Guthrie, he 
believes such contiguous conditioning to occur only in the 
neighborhood of a reinforcing state of affairs. This has two 
consequences in Hulls theory. A stimulus on the one hand 
tends to become more strongly associated with responses made 
immediately, because the stimulus-trace is stronger at first. 
Its tendency to become associated with responses occurring 
later in the series is correspondingly less because the trace is 
weaker. On the other hand, the later responses (particularly 
goal responses) are nearer to the reinforcing state of affairs, 
and so even a weaker trace will become more strongly con- 
ditioned to them, according to the gradient of reward. Hence 
it is that stimuli, such as those associated with an alley on the 
way to food, tend to evoke both the running behavior appro- 
priate to that alley and fractional anticipatory goal responses. 
At a distance from the goal the running gets conditioned be- 
cause the stimulus-trace is strong, even though the reinforce- 
ment is weak; the goal Tesponses get conditioned to remote 
stimuli even though the trace is weak because the reinforce- 
ment is strong. 

Any conjectures regarding the factors underlying the vari- 
ous gradients of reinforcement require empirical verification 
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before they can be accepted as established explanations. 
Theories of strict simultaneity of associated events tend to 
ignore the directional character of behavior. Events may be 
touching and yet be organized in a series in which one event 
precedes another. The effectiveness of the forward order in 
conditioning is probably related to a fundamental directional 
organization of behavior. 


Summary 


Any conditioning experiment involves a number of events 
beginning with the conditioned stimulus and ending (for any 
one trial) with the reinforcement. In terms of effectiveness for 
conditioning there are more favorable and less favorable inter- 
vals between sequential events. Since effectiveness tends to 
diminish gradually with increasing (or decreasing) interval, 
the relationship between strength of conditioning and interval 
may be expressed as an empirical gradient. Depending upon 
the arrangements of the experiment, and the place in the 
sequence of events at which the interval is systematically 
changed, a number of different gradients may be described, 
such as the temporal gradients within classical conditioning 
and within avoidance learning, and the gradient of reward 
within instrumental learning. Since the interval is usually 
terminated by reinforcement, the gradients have been classed 
together as gradients of reinforcement. 

The goal gradient is closely related to the gradients of re- 
ward and of avoidance. It is a result of learning in which the 
principles underlying the other gradients have been important, 
but it is not a simple expression of these other gradients. In 
the first place, the goal gradient always refers to a situation 
in which a number of serial acts have been reinforced by a 
common goal situation. In the second place, the acts measured 
in determining the form of the goal gradient are not discrete 
conditioned responses, but are parts of a larger behavior pat- 
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tern in which separate acts are submerged, and in which fac- 
tors other than the gradient of reinforcement are of signifi- 


cance. 


The various gradients may be accepted in their empirically 
determined forms, or the attempt may be made to account for 
them on the basis of simultaneous conditioning by inferring 
some sort of trace or intervening process to span the interval 
between the events being studied. 


Notes 


Temporal Gradient in Classi- 
cal Conditioning 


On favorable forward orders of 
a few tenths of a second, cf. Hil- 
gard (1931), Schlosberg (1928), 
Wendt (1930). On the unfavorable- 
ness of intervals near strict simul- 
taneity, cf. Hilgard and Biel (1937). 
On backward conditioning, cf. Ca- 
son (1935), V. I. Pavlova (1933), 
Porter (1938b), Pressman (1934), 
Switzer (1930). Only Switzer re- 
ports much success, and it is pos- 
sible that his results may be a form 
of sensitization or pseudo-condition- 
ing (Bernstein, 1934; Grether, 
1938). 

For discussion of minimum la- 


tency, cf. Cason (1922b, 1934a), 
Culler (1938b), Hilgard (1931, 
1934), Wendt (1930), H. M. 
Wolfle (1933). The direction of 


latency changes in delayed and 
trace conditioning is discussed by 
Hull (1929), Pavlov (1927, p. 89), 


Rodnick (1937a), Steckle (1933), 
Switzer (1933). 


Gradient of Reward 


On delayed reward, cf. Clements 
(1928) Re Te Hamilton (1929), 


Roberts (1930), Warden and Haas 
(1927), Watson (1917), Wolfe 
(1934), A. B. Wood (1933). Ex- 
periments on time discrimination in 
animals often depend upon the 
greater efficacy for learning of that 
segment of the path which involves 
the least delay, and hence they rep- 
resent forms of experimentation on 
the temporal gradient of reward. 
Among these are experiments by 
A. C. Anderson (1932), Finan 
(1939), Sams and Tolman (1925). 

The ‘spread of effect,’ which is 
Thorndike’s statement of the gradi- 
ent of reward, was announced first 
in a note in Science (1933a), and 
elaborated in subsequent publica- 
tions (1933b, 1935). Because a 
number of acts are reinforced by a 
single reward, the ‘spread of effect’ 
situation is comparable to the goal 
gradient (Hull, 1932). The goal 
gradient principle was not at first 
coordinated with the law of effect, 
but the similarity has since been 
recognized and pointed out by Hull 
(1935d, 1937), Brandt (1935), 
Muenzinger, Dove, and Bernstone 
(1937). It may be noted also that 
the verbal type of reward in the 
Thorndike experiment is a second- 


GRADIENTS OF REINFORCEMENT 


ary or derived reward, so that re- 
inforcement, according to our class- 
ification, is of the derived form. 


Goal Gradient 


The expression ‘gradient of re- 
inforcement’ was first used by N. 
E. Miller and Miles (1935) to refer 
to the fact that acts more remote 
from reinforcement were strength- 
ened by conditioning less than acts 
near to reinforcement. They pro- 
posed essentially the same distinc- 
tion which we have adopted between 
this gradient and the goal gradient 
which tends to be produced by it, 
the goal gradient being one of 
strength of response rather than of 
strength of conditioning. Deduc- 
tions based on the goal gradient 
hypothesis, and their experimental 
tests, are discussed in Chapter 9. 
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Contiguity 


For general discussions of time 
interval in association and condition- 
ing see Guthrie (1933), Robinson 
(1932), Wenrick (1933). Wenrick 
finds chaos, which Guthrie resolves 
through the notion of associative 
mediation, so that according to him 
all conditioning is actually simul- 
taneous. Robinson, admitting that 
associative mediation cannot be dis- 
proved, believes it more useful to 
accept the empirical fact that asso- 
ciative strength appears to be a con- 
tinuous function of time interval, 
and hence he puts the burden of 
proof upon those who insist upon 
absolute simultaneity as basic for 
association. The empirical gradients 
may be accepted as valid, which- 
ever position is adopted. 


Chapter 8 
GENERALIZATION AND DISCRIMINATION 


HEN an organism has learned to give a conditioned re- 
\ \ sponse to a particular stimulus it can be shown that 
other similar stimuli will also elicit the response even though 
these other stimuli have not been used in the conditioning 
experiment. The partial equivalence of different stimuli in 
evoking a conditioned response is known as sensory generali- 
zation. There is also a degree of equivalence among responses 
which may be called response generalization, so that a stimulus 
which has come through training to elicit a particular response 
may, under some circumstances, elicit a different response 
without special training. It is evident that generalization is 
important in the adaptive economy of the organism because 
environmental situations never recur in nature without 
changes. If situations had to be repeated in identical form in 
order that previous experience might be effective, learned 
adjustments would be of little service. On the other hand, 
generalization must in many cases be restricted to a narrow 
range of stimuli and responses, as the organism learns in its 
natural environment to react selectively to certain stimuli or 
aspects of stimuli and not to others. Selective reaction is the 
opposite of generalization: it involves discrimination of stimuli 
and corresponding differentiation of response. 
The basic facts of stimulus equivalence and of response 
equivalence are not limited in application to conditioned re- 
sponses, but are true of reflexes and of complex voluntary 


responses. Every response is elicitable, not just by one stimu- 
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lus, but by a class of similar stimuli. Correspondingly, every 
stimulus elicits, not just one response, but one of a class of 
responses. In the case of unlearned reflexes, the degree of 
generalization is determined by the inherited structural organ- 
ization of the nervous system. With learned responses, how- 
ever, the degree of generalization is modifiable through ex- 
perience. The study of generalization and discrimination in 
conditioned responses offers the possibility of determining the 
fundamental principles of such modification in simple, ex- 
perimentally controlled situations. 


Sensory Generalization 


Generalization to Stimuli Differing in Quality and In- 
tensity. The experiments of Hovland on the generalization 
of a conditioned galvanic skin response to tones differing in 
pitch or in loudness serve as convenient illustrations of gen- 
eralization along dimensions within a sensory modality. 

For the generalization of pitch (1937a), four tones were 
selected by psychophysical methods to be separated in pitch 
by 25 just noticeable differences (j.n.d.’s). The frequencies 
chosen were 153, 468, 1000, 1937 cycles. The tones, having 
been equated for loudness, now represented a scale of approxi- 
mately equal units on a dimension of pitch similarity, as 
judged by the verbal reports of the subjects. One group of 
subjects was conditioned to the lowest tone, and then tested 
on the others to determine the magnitude of the generalized 
responses. A second group was conditioned to the highest tone, 
and tested with the three lower tones. Permutational orders 
of presentation made possible the pooling of the results of all 
subjects into a single curve representing the generalization of 
tones separated in pitch by 25, 50, and 75 j.n.d.’s. The curve, 
reproduced in Figure 18, shows that the degree of generaliza- 
tion decreases progressively with greater differences in pitch, 
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producing, when plotted, a concave (decelerated) gradient. 
Humphreys (1939c) has extended Hovland’s observations by 
measuring the generalization to tones differing by only 5 and 
15 j.n.d.’s. Since the greatest decrease in the gradient of gen- 
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FIGURE 18. GENERALIZATION oF Prrcu: conpITIONING. The abscissa is in 
units of 25 j.n.d.’s. The actual frequencies involved were 153, 468, 1000, and 
1967 cycles, equated for loudness. Half of the 20 subjects were conditioned 


to I z half to 1967 cycles, and tested at all four frequencies. (Hovland, 
1937a. 


FIGURE 19. GENERALIZATION 
reinforced, so that the condition 
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oF PrrcH: EXTINCTION. All frequencies were 
ed responses to all were similar in amplitude. 
e extreme frequencies were extinguished, and the others tested. The 
e represents the amount by which the response as tested was less than 


c aaah response at the end of training. Modified from Hovland 
19374). 


} FIGURE 20. GENERALIZATION oF LoupNEss: 
is In units of 50 j.n.d.’s. All tones wer 
decibels above threshold. Half of the 3 
half at 86 db., then tested at all four 


CONDITIONING. The abscissa 
e 1000 cycles, at 40, 60, 74, and 8 

2 subjects were conditioned at 40 db., 
intensities. (Hovland, 1937b.) 


eralization occurred at the first interval, it may be concluded 
that the gradient is concave, even for stimuli differing only 
slightly from the conditioned stimulus. 

The generalization of extinction was studied by Hovland in 
other subjects after conditioned responses were established 
equally to the four frequencies. Responses of half the subjects 
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were then extinguished by presenting the highest tone with- 
out reinforcement, the other half by presenting the lowest 
tone. Generalization of extinction was determined by the 
amount of reduction of response to the other frequencies. The 
curve of generalization of extinction was found to be similar 
to that of the generalization of conditioning, as shown in Fig- 
ure 19. The important features of both curves are (1) a large 
amount of generalization even with gross differences in fre- 
quency, and (2) a concave (decelerated) gradient in the mag- 
nitude of generalized responses with increasing difference in 
pitch from the stimulus conditioned. 

A similar experiment was performed for the generalization 
and extinction of responses based on the intensity of tones 
(Hovland, 1937b). Four tones of 1000 cycles were selected, 
spaced at intensities corresponding to 50 j.n.d.’s in loudness. 
The curve of generalization, while similar to that for pitch, 
is much flatter; that is, generalization is greater for intensity 
than for frequency (Figure 20). 

Generalization to Spatially Separated Stimuli. Another 
form of generalization is that in which the stimuli remain 
qualitatively alike, but are presented at different places on 
the skin, so that the sensory effects differ in location. This is 
the arrangement used in the classical studies of ‘irradiation’ 
in Pavlov’s laboratory (Pavlov, 1927, p. 152ff.). The sensory 
consequences of the stimulation of neighboring points are 
presumably more similar than the consequences of stimulating 
remote points. Anrep (1923), using the method of conditioned 
inhibition, found a spatial gradient of generalization for the 
dog similar to that found by Hovland, provided points on the 
dog’s skin were tested immediately after stimulation with the 
inhibitory combination. Spatial gradients of generalization of 
both conditioning and extinction were reported in man by 
Bass and Hull (1934), using the conditioned galvanic skin 
response based on electric shock reinforcement. Vibro-tactual 
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stimulators were placed one on the back near the shoulder, 
one on the left side of the small of the back, one on the left 
thigh, and one on the left calf. When a conditioned response 
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Ficure 21 
STRENGTH oF GENERALIZED Responses DURING THE COURSE OF 
CONDITIONING 


The solid line represents the amplitude of the galvanic skin response to 
the conditioned stimulus at successive stages of conditioning. The broken 
ine represents the amplitude of the same response to the generalized 


stimulus. (From Hovland, 1937d.) 
was established to stimulation of the point on the shoulder 
(or on the calf), the response generalized to the other stimu- 
lators, progressively less as distance increased. They reported 
the gradient of conditioning to be convex, that of extinction 
to be concave. 

The concave gradients found by Hovland and by Hum- 
phreys may be accepted as of more general significance, since 
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their stimuli were arranged in a single dimension of equated 
distances. E. J. Gibson (1939) has reported a gradient of 
similar form for voluntary responses. The precise form of the 
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RELATIVE Amount or GENERALIZATION DURING THE COURSE OF 
ConDITIONING 


_ The data of Figure 21 have been replotted to show the percent of general- 
ization at successive stages of conditioning. Each point on the curve rep- 
resents the ratio of the strength of generalized responses to the strength of 
conditioned responses. 


gradient may depend on other facts in the testing situation 
than those considered above. Its relation to degree of condi- 
tioning and of extinction has not been measured although 
Changes may be expected with practice, as indicated in the 
hext section. Anrep (1923) has reported that the form of 
gradient depends on the time interval elapsing between the 
training stimulus and the stimulus which tests the generaliza- 
tion, but these results have been questioned by Loucks (1933). 

Generalization as a Function of the Strength of Con- 
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ditioning. As reinforcement is repeated, generalization in- 
creases gradually to a maximum, and then begins to decrease 
(Beritov, 1924, p. 123). Another of Hovland’s experiments 
(1937d) presents quantitative data on this point. In Figure 21 
are given corresponding curves of the amplitude of reinforced 
and of generalized responses at different stages of practice. It 
is evident that as reinforcement continues, both responses in- 
crease, but at different rates. The curve for the generalized 
response flattens earlier. The relationship between the two 
responses is shown in Figure 22. Here the generalized re- 
sponse is expressed as a percentage of the reinforced condi- 
tioned response. While the absolute amount of generalization 
increases throughout the period of practice studied, the rela- 
tive amount of generalization reaches a maximum and then 
declines slightly. 

Great caution is required before the results from isolated 
situations can be applied to other experimental arrangements 
that introduce complications which have not been tested em- 
pirically. This is shown in some findings of Razran (1939e) 
regarding the effects of repeated reinforcement on generaliza- 
tion. He conditioned salivary responses of human subjects, 
using lights as conditioned stimuli. When single lights were 
used, other lights of different color showed generalization 
similar to that reported by Hovland. There was an initial rise 
in the per cent of generalization, followed by a decrease, as 
shown in the solid curve of Figure 23. However, when patterns 
of lights were used as conditioned stimuli, Razran found that 
the amount of generalization increased progressively with 
Practice, as shown in the broken curve of Figure 23. 

The Extinction of Responses Elicited Through Sen- 
sory Generalization. Some interest centers in the ease with 
which generalized responses can be extinguished, for this bears 
upon the rapidity with which a discrimination can be estab- 
lished. Hovland (1937c) has reported data on this problem. 
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After 16 presentations of a vibrator followed by a shock, when 
the generalization should be at its maximum, the conditioned 
galvanic response was tested both by the stimulus used in 
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EXPERIMENTAL PERIODS 
FIGURE 23 
GENERALIZATION WITH SINGLE STIMULI AND WITH PATTERNS OF STIMULI 


Generalization of salivary conditioned responses in 16 human subjects. 
The reinforced conditioned stimuli were flashings of red and green lights; 
the tested stimuli blue and yellow lights. Lights were presented either 
singly, or in geometrical patterns, with the differences as indicated. Percent 
generalization is derived by dividing the mean salivation to the equivalent 
stimulus by the mean salivation to the reinforced stimulus. (Plotted from 
the data of Razran, 1939e, Table 1.) 


training, and by one differing in intensity. Half of the sub- 
jects were trained with each stimulus, to control the effect of 
the difference in strength of stimuli. The conditioned responses 
to the reinforced stimulus were found to extinguish more 
slowly than the generalized conditioned responses to the other 
stimulus. After four extinction trials, the conditioned re- 
sponse remained at 89 percent of its original magnitude, while 
the generalized response had fallen to 60 percent of its original 
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magnitude. The more rapid extinction of the generalized re- 
sponse was followed by greater spontaneous recovery, so that 
24 hours after extinction the difference between generalized 
and conditioned responses was no longer statistically signifi- 
cant. The more rapid extinction of generalized conditioned re- 
sponses had been noted also by Liddell, W. T. James, and O. D. 
Anderson (1934). 

Response Generalization. It has been pointed out that 
an organism which has learned to respond in a certain way 
to stimulus A, has thereby learned also to respond in a similar 
way to an equivalent stimulus A’. The counterpart on the 
response side is as follows: if an organism has learned to 
react with response B to a stimulus A, it has also learned 
thereby to react with a response B’, which is unlike B, but in 
some respects equivalent to it. One illustration comes from 
Bekhtereyv (1932, p. 216) who found that a dog lifted another 
foot to the conditioned stimulus when the one which normally 
responded was fastened down. Another instance is furnished 
by Lashley’s (1924) monkey, which solved a problem of 
manipulation with the other hand after the hand which had 
always been employed previously was paralyzed due to a 
brain operation. Further observations of the kind suggested 
by Bekhterev have been reported by Kellogg (1939b). Fol- 
lowing buzzer-shock conditioning in which the shock was 
applied only to the right hind foot of a dog, it was found that 
each of the four legs occasionally responded to the buzzer. 
Responses in the more remote legs were less frequent than 
those in the shocked leg, the relative frequencies of condi- 
tioned responses being in the order right hind leg (the one 
shocked), left hind leg, right foreleg, left foreleg. This repre- 
Sents a crude generalization of response with remoteness from 
reinforcement, corresponding somewhat to the sensory gradi- 
<nis previously, described. Since most ot the melee Ole: 
sponse generalization which have been studied in relation to 
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conditioning involve a transfer of training based on previous 
learning, their consideration is postponed to Chapter Io. 


Discrimination 


Every conditioned response involves discrimination to 
some degree. A response to a particular sound is not evoked 
by all sounds, and sometimes not by the same sound presented 
in different positions (Girden, 1938). The experimental prob- 
lems of discrimination arise, however, when there has been a 
prior generalization, so that stimuli are responded to as equiv- 
alent, and subsequently it is desired to secure a discrimination 
between them. To the extent that generalization is incomplete, 
there is already a partial differentiation of response, but it is 
often desired to increase this in order that the limits of dis- 
crimination may be tested. 

Discrimination in Classical Conditioning. Although re- 
peated reinforcement reduces generalization, it never results 
in very fine differentiation. In order to produce finer dis- 
crimination the generalized responses are extinguished by 
non-reinforcement, while reinforcement of the conditioned re- 
sponse continues. This is known as the method of contrasts, 
and has been used widely by Pavlov and by other students 
of sensory processes. 

Specimen curves showing the course of conditioned dis- 
crimination in man are reproduced in Figure 24. In the experi- 
ment illustrated, the negative stimulus was introduced only 
after conditioned responses to the positive stimulus were well 
established. The discrimination plotted was between two ad- 
jacent illuminated apertures, easily distinguished by the sub- 
jects, so that threshold capacity was not involved. 

Discrimination in Instrumental Conditioning. The 
classical discrimination experiment of Pavlov is carried out 
by presenting the positive and the negative stimuli succes- 
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sively rather than simultaneously. The same procedure may 
be used equally well with instrumental responses. When the 
positive stimulus is presented, the proper response is re- 
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Course OF DISCRIMINATION BY THE METHOD OF CONTRASTS 


The curves represent mean values of the conditioned eyelid responses 
of 10 human subjects. The positive stimulus was the illumination of one 
aperture, followed by an air-puff to the cornea. The conditioned response 
to the positive stimulus was established on the first day. The negative stim- 
ulus, introduced in random alternation with the positive stimulus on the 
second day, consisted in the illumination of an adjacent aperture. The nega- 
tive stimulus was never reinforced. When the negative stimulus was intro- 


duced there was at first generalizatio i incti 
, th . (From 
Hilgard, R. K. Campbell, and W. N. see ea ae 


warded; when the negative stimulus is presented, no reinforce- 
ment is given, and in some cases, punishment may be admin- 
istered if the response occurs. Experiments of essentially this 
kind have been carried out by Elder (1934, 1935) and by 
Wendt (1934). Elder taught chimpanzees to react to a tele- 
graph key following a readv signal, provided a tone was 
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heard. The animals learned to refrain from response after the 
ready signal when the tone was absent. Wendt’s method was 
similar. Monkeys learned to open a drawer which contained 
food following the positive auditory stimulus. The drawer was 
locked following the negative (i.e., absent) stimulus, and the 
animal learned to withhold reaction. 

Skinner (1933, 1938) has reported two types of discrimina- 
tion experiment using the lever-pressing situation. In the first, 
similar to the experiments just described, lever-pressing is re- 
warded in the presence of a differential cue, such as a light, 
and non-rewarded when this cue is absent. In the second type 
of experiment, a differential characteristic of lever-pressing is 
rewarded, 1.e., only responses above a certain force will secure 
food. Skinner makes the distinction between the rat’s dis- 
criminating among environmental stimuli, and differentiating 
among its own responses. It can, of course, be taught either 
type of discrimination, as choice-reaction experiments illus- 
trate. 

Choice-Reaction Discrimination. Discrimination learn- 
ing is commonly studied in experimental situations in which 
the positive and the negative stimuli are presented simulta- 
neously, but in differing spatial relationship. For example if 
the positive stimulus is on the right side, the animal is re- 
warded when it makes the proper response such as proceeding 
to the right side. If the animal makes the wrong response, 
that is, goes to the left, no reward is given, and punishment 
may be administered. On succeeding trials, the positive stim- 
ulus is presented in random order on the right or left side 
until the animal has learned always to proceed to the side 
corresponding to the positive stimulus. Many different kinds 
of reaction have been used in this type of discrimination learn- 
ing. The subject may be trained to pull a string on the left 
or right which will bring food if correct (Kliiver, 1933), to 
open one of two possible boxes to secure food (Spence, 1934), 
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to jump through an opening (Lashley, 1930b), to push opena 
door (Munn, 1931), and so forth. 

There are three important differences between the choice- 
reaction and the Pavlovian procedures for studying discrimi- 
nation learning. (1) The two stimuli are presented simul- 
taneously rather than separately on successive trials. The re- 
lational characteristics of the stimuli are presumably more 
apparent when the stimuli are present together. (2) The 
choice is between two reactions rather than between reaction 
and restraint. Each of the reactions may have a double deter- 
mination, being an approach to one of the stimuli and a re- 
action away from the other. (3) The subject makes a choice 
on every trial, since reinforcement or punishment is given only 
after the reaction has occurred. In the Pavlovian procedure, 
reinforcement by the unconditioned stimulus is presented 
after each occurrence of the positive stimulus regardless of 
the reaction made. | 

These differences will be very important in the empirical 
problems of the speed or limit of any particular discrimina- 
tion, and must be considered in the formulation of a general 
explanation of discrimination. 

Mechanism of Discrimination Learning. The simplest 
explanation of the process of formation of a differential re- 
sponse to two stimuli is in terms of the reinforcement of one 
conditioned response and the extinction of another response 
which was previously learned or which was generalized from 
the reinforcement. These two modifications cannot proceed 
independently, however, for each will have an effect upon the 
other. Strengthening of the response to the positive stimulus 
will automatically strengthen the response to the negative 
stimulus through generalization. Similarly, weakening of the 
response to the negative stimulus by extinction will simulta- 
neously tend to weaken the positive response through general- 
ization of extinction. It is apparent that if generalization were 
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complete, as would occur if the two stimuli were identical, no 
progress in discrimination would take place. To the extent that 
the two stimuli are dissimilar, however, the primary effect of 
reinforcement or extinction will be greater than the general- 
ized effect, and after a sufficient number of repetitions, the dis- 
crimination will be perfect. The number of necessary repeti- 
tions will increase as the degree of generalization increases. 
This is illustrated in the familiar fact that it is more difficult 
to establish a discrimination than a simple conditioned re- 
sponse, and that the difficulty of a discrimination is a func- 
tion of the similarity of the two stimuli employed. 

The first of several careful attempts to treat the discrimina- 
tion experiment quantitatively in terms of conditioning prin- 
ciples was undertaken by Spence (1936). On the familiar 
conditioning assumptions that reinforcement results in a cu- 
mulative increase in reaction tendencies and non-reinforce- 
ment in a cumulative decrease, Spence was able to deduce 
several of the most prominent characteristics of discrimination 
learning. 

The first of these deductions concerns the behavior of the 
animal during the early trials of discrimination training. 
Krechevsky (1932b) carried out a series of investigations with 
the white rat in which he showed that the reactions of the 
animal during the pre-solution period were not a chance affair 
but consisted of systematic modes of response such as posi- 
tion and alternation habits. He termed these systematic pre- 
solution responses ‘hypotheses’, and interpreted them as at- 
tempted solutions of the problem by the rat. Spence (1936) 
set up a hypothetical situation in which he was able to derive 
such non-random behavior from the assumption of cumulative 
strengthening and weakening of the tendencies to the two 
stimuli. Krechevsky’s conception implies that the training 
during the pre-solution period has no influence in determin- 
ing the ultimate correct solution except to eliminate incorrect 
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hypotheses. Spence’s theory, on the other hand, demands 
that each reinforced or non-reinforced trial contribute to the 
strength of the response tendency. A critical experiment would 
be one determining the effect of reversal of the positive and 
negative stimulus values during the pre-solution training. T. 
L. McCulloch and Pratt (1934) found that such a reversal 
significantly retarded the learning of the correct solution, 
while Krechevsky (1938) found that there was no effect. 
Further experimental data are necessary to resolve this dis- 
crepancy. 

Another feature of discrimination learning is the frequent 
occurrence, at least in the higher animals, of sudden solutions 
following periods of apparently chance response. This phe- 
nomenon has been emphasized by those who propose explana- 
tions in terms of ‘insight’ or ‘reorganization of the sensory 
field.” Spence (1938), however, found that analysis of the 
learning curves of chimpanzees indicated that sudden learn- 
ing, like gradual, was closely correlated with the relative 
strengths of the tendencies to respond to the two stimuli as 
determined by the number of previous reinforcements and 
non-reinforcements. Sudden solutions were more likely to 
occur in animals which were fast learners as compared with 
slow learners, since the increment in strength of condition- 
ing resulting from each reinforcement was inferred to be 
greater. 

A third problem which has been prominent in discussions of 
discrimination learning is concerned with the question of re- 
sponse to relative as contrasted with absolute properties of the 
stimuli. It is a typical finding that an animal which has been 
trained to select the brighter of two lights will select the 
dimmer one if it is presented together with a still dimmer 
light. Gestalt arguments against association theories have 
frequently cited transposition experiments, in which stimuli 
are equivalent by virtue of relationships of pattern, bright- 
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ness, or other ‘transposable’ properties. By adding some 
plausible assumptions with respect to the quantitative aspects 
of generalization of conditioned responses to stimuli varying 
in a single dimension, Spence (1937b) has deduced types of 
discrimination which are usually considered to be based on 
the relational character of the stimuli. The nature of Spence’s 
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FIGURE 25 
Spence’s THEORETICAL REPRESENTATION OF DISCRIMINATION 


Areas of the visual stimuli used in the experiment are represented along 
the baseline. During the training, response to 256 is rewarded, response to 
160, extinguished. As a result of reinforcing 256 the generalization repre- 
sented by the solid lines results. That is, tendencies to react positively are 
generalized to the neighboring stimuli in proportion to the heights of the 
solid lines above each stimulus size. The non-reinforcement of 160 leads to a 
generalization of inhibition represented by the broken lines, extending also 
to stimuli on either side of 160. The resulting reaction tendencies, obtained 
by subtracting the negative from the positive, are indicated by the figures 
above each stimulus size. (Spence, 1937b.) 

_ The use of the diagram to explain the results of transposition experiments 
is described in the text. 


analysis is illustrated by Figure 25, which is reproduced from 
his paper. As indicated in the caption to the figure, the gradi- 
ents of conditioning and of extinction are set up by training 
with stimulus 256 as positive (reinforced) and stimulus 160 
as Negative (non-reinforced). Suppose now that a test of 
transposition is made with the pair of stimuli 409 and 256. 
From the diagram it is evident that the strength of the tend- 
ency to react to 409 (6.5) is greater than the tendency to 
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react to 256 (4.8). Hence 409 will be reacted to, in spite of 
the fact that 256 was the positive stimulus during training. 
For this pair of stimuli, results conform to reaction to ‘rela- 
tively larger’ although no assumption is made that the or- 
ganism detects such a relationship. Suppose, however, the pair 
655 and 1049 are presented. Now the strengths are 6.7 for 
655 and 2.5 for 1049. Here the reaction will be to the smaller 
of the two stimuli, contrary to the prediction based on re- 
action to relations. The crucial test supporting Spence’s theory 
lies in the failures of transposition when tests are made with 
stimuli remote from those used in training. Such failures have 
indeed been found in experiments by Gulliksen (1932), 
Kliiver (1933), and Spence (19372). 

More recently, Hull (1939b) has presented a discussion of 
stimulus equivalence which represents an elaboration and re- 
finement of the type of argument used by Spence. He has 
adopted a concave gradient of generalization instead of the 
convex one illustrated in Figure 25, and has expressed the 
consequences of the interaction of positive and negative tend- 
encies of the two gradients in precise mathematical form. 
Another mathematical theory of discrimination learning has 
been elaborated by Gulliksen and D. L. Wolfle (1938a, 1938). 
The theories of Spence, Hull, and Gulliksen and Wolfle in 
spite of differences in detail have in common the assumptions 
(1) that every reinforcement leads to an increment in the 
tendency to repeat the behavior in the presence of the stimuli 
preceding reinforcement, (2) that every non-reinforcement 
leads to a decrease in the tendency to react again to the stimuli 
which preceded non-reinforcement, (3) that the tendencies 
are generalized to other stimuli along dimensions defined 
psychophysically, (4) that these reaction tendencies, both in- 
creases and decreases, interact algebraically, (5) that dis- 
PEN REN 0, reactions are based on the resolution of the 
competing tendencies in favor of reaction to that stimulus 
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which has the greater excess of positive reaction tendencies 
conditioned to it. 

This formulation of the problem of discrimination does not 
represent a final theory in any sense. Whatever revisions are 
made in the future, the papers discussed above represent im- 
portant advances in clarifying assumptions, delineating im- 
plications, framing propositions for experimental testing, and 
providing a common mathematical structure for diverse phe- 
nomena. Limitations are suggested at the present time by 
the observed difference in degree of generalization when pat- 
terns of stimuli are involved (Figure 23), and by the results 
of experiments on partial reinforcement (Chapter 6). The 
influence of partial reinforcement in the discrimination situa- 
tion has been studied experimentally by Brunswik (1939). 
Rats running a simple T-maze, with but a single choice-point, 
were fed occasionally on the right, occasionally on the left. 
Brunswik found that choices came to be made either to the 
right or left roughly in proportion to the number of trials on 
which reinforcement had been received on one side or the 
other (Table 6). This behavior, which Brunswik interprets as 
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reaction in accordance with the probability of reward (hence 
in terms of the animal’s hypotheses), would be interpreted by 
Spence in accordance with a calculus of reinforcement and 
extinction. That is, in Spence’s analysis each reward on the 
right would strengthen the tendency to turn right, each ab- 
sence of reward would weaken the tendency. The probability 
theory of Brunswik is a sign-learning explanation of the in- 
fluence of past experience on present performance, and as 
such is an important alternative to a conception of simple 
summation of reinforcement and extinction. 

A simple probability theory arrives at predictions very 
similar to those based on reinforcement and extinction, since 
every strengthening by reinforcement is at the same time an 
increase in the probability that the stimulus will be reinforced 
again if reacted to. It has been shown by Humphreys (1939a, 
1939b), however, that a simple probability theory does not 
suffice, since expectancies are complicated by patterns of past 
successes and failures as well as by the total number of experi- 
ences of reward and non-reward. He shows conclusively for 
human subjects that a single non-reinforcement following a 
pattern of uniform reinforcement leads to the expectation of 
uniform non-reinforcement, while following a pattern of ran- 
dom reinforcement and non-reinforcement the expectation of 
uniform non-reinforcement requires several non-reinforced 
trials. His interpretation permits him to account for the more 
gradual extinction found in both human subjects and rats 
following random alternation of reinforcement and non-re- 
inforcement. The corrections which Humphreys suggests ap- 
ply both to probability theories and to theories based on the 
summation of reinforcement and extinction. In its present 
form the probability theory espoused by Brunswik (1939) 
makes more allowance for these corrections than the rein- 


forcement-extinction proposals of Spence (1936) and of Hull 
(1939b). 
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Threshold Measurement. Several different procedures 
for the establishment of conditioned discrimination have been 
described above. To these must be added the usual psycho- 
physical methods employing instructed verbal response with 
human adult subjects. The chief interest in the comparison 
of the various methods lies in the measurement of thresholds. 
Many problems require the determination of absolute and of 
differential thresholds in animals where verbal response is not 
possible. 

In man, thresholds obtained by conditioning methods may 
be compared with those obtained by the more usual psycho- 
physical judgments. Newhall and R. R. Sears (1933) com- 
pared absolute visual thresholds obtained simultaneously by 
verbal report and by conditioned finger withdrawal. Using 
the method of constant stimuli they found that the thresholds 
by the two techniques were similar. Neet (1936) compared 
several conditioning methods. He found that conditioned eye- 
lid reactions yielded auditory thresholds similar to those ob- 
tained by verbal report; manual response and conditioned 
respiratory responses yielded slightly higher thresholds. A 
threshold is therefore relative to the method chosen for meas- 
uring it. That conditioning may occur to subliminal stimuli 
is not necessarily contradictory to the reports that the thres- 
holds by conditioning and other methods are equivalent, for 
by definition of the limen in the method of constant stimuli, 
a certain number of correct responses must be given to ‘sub- 
liminal’ stimuli. 

In animals, thresholds obtained by classical conditioning 
methods may be compared with those obtained by other 
methods, such as the discrimination box of Yerkes and Wat- 
son (1911), or the electrical responses of the cochlea (Stevens, 
H. Davis and Lurie, 1935). Conditioning methods appear to 
be somewhat more sensitive than the original locomotor form 
of the discrimination box. This was suggested by Anrep’s 
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(1920) greater success with tonal discrimination in dogs than 
Johnson’s (1913). Subsequently, discrimination procedures 
involving instrumental response have been greatly improved 
by arrangements for presenting the reinforcement immediately 
following the animal’s reaction, without the long delay in- 
herent in the Yerkes-Watson apparatus. This change is illus- 
trated in Lashley’s (1930b) method, in which the stimuli to be 
discriminated are mounted on doors against which the animal 
must jump across an intervening space. If it jumps to the 
negative stimulus it finds the door locked, and falls into a net. 
If the positive stimulus is selected, the door gives way under 
the force of the jump, and the animal finds food on the plat- 
form behind the door. The immediate reinforcement pro- 
vided by this procedure results in more rapid discrimination 
learning and in more precise threshold values. 

The value of conditioning and closely related procedures 
in the conventional problems of sense-physiology may be illus- 
trated by some of the recent advances in audition and vision. 
Localization of tonal frequencies in the cochlea, substantiating 
certain aspects of the Helmholtz resonance theory of pitch, 
has recently been found by several methods. The electrical 
methods agree with the results of conditioning experiments 
(Culler, Willmann, and Mettler, 1937). The duality of the vis- 
ual process in the rabbit, represented by a shift of the bright 
visibility curve by approximately 30 mu toward the red end 
of the spectrum from that of dim visibility has been demon- 
strated by R. H. Brown (1936a, 1937) by means of condi- 
tioned respiration. Using the method of contrasts, he also es- 
tablished a differential response to two hues of equal visibility 
(R. H. Brown, 1936b). This is the first convincing demonstra- 
tion of color vision in mammals below the primates. 

Discrimination as a threshold method depends upon experi- 
mental arrangements such that the animal reacts differentially 
to the minimum stimulus difference which it can perceive. 
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Successful differential response secured under proper experi- 
mental conditions guarantees that the animal can discrimi- 
nate the relevant differences; unsuccessful discrimination does 
not, however, mean that the differences involved are beyond 
the perceptual capacities of the organism, for with some other 
experimental arrangement the differentiation might be found. 
Failure of discrimination may be due, for example, to in- 
sufficient training. Instances in which an entire year of train- 
ing has been found necessary are reported (Liddell, W. T. 
James, and O. D. Anderson, 1934). In other cases, the animal 
may become upset and refuse to cooperate at all in the experi- 
mental procedure. Failure to demonstrate capacity in per- 
formance may mean that relationships within the situation, 
other than the perceptual ones, are too difficult for the animal. 
The history of animal experimentation contains many in- 
stances in which modified methods have resulted in modified 
thresholds. In man, conditioned discrimination has been 
shown to depend on factors other than the capacity to dis- 
criminate between the stimuli, factors such as time interval 
between stimulus and reinforcement, knowledge of stimulus 
relationships, and instructions with respect to response. 


Similarity and Difference as 
Experimental Problems 


Generalization and differentiation are the processes within 
conditioning which throw light on perception, and perception 
historically has been one of the important fields of psychology. 
Some of the theoretical problems arising out of the equivalence 
of stimuli are particularly important, because interpretations 
of perceptual organization have been the bulwark of the 
Gestalt attack on the types of learning theory among which 
conditioning theories are included. To the extent that a per- 
ceptual whole is more than the sum of its parts in other rela- 
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tionships, it may also be more than the sum of its parts in 
conditioning experiments. As a method for detecting equiv- 
alences, conditioning may be expected to disclose results 
similar to those obtained by other methods. 

Patterns of Simultaneous Stimuli. When a combination 
of stimuli has been regularly reinforced, the component stim- 
uli are not equally effective in eliciting responses when pre- 
sented alone. Pavlov (1927, 141ff.) reports that one member 
of the compound stimulus may elicit responses of full strength, 
while others remain ineffective, especially if parts of the com- 
pound are presented to different senses. Razran (1938d and 
later) has performed a number of studies in configural con- 
ditioning. One of his methods tested salivary conditioning to 
the component lights of a pattern which the human subjects 
had watched while eating. In order to avoid the disturbing 
effect of voluntary attitudes, the experiment was represented 
to the subject as concerned with the effect of eye-fatigue on 
digestion. The results show a tendency for a gradual shift of 
effectiveness from one component part of the whole to an- 
other. In general, that part which was at first effective was 
relatively simple (a strong Gestalt), while the effective part 
after many reinforcements was more complex (a weaker 
Gestalt). This is consonant with the restriction of generaliza- 
tion with practice, for the more complex part actually repre- 
sented the particular pattern more specifically than the 
simpler parts at first responded to. Since the organization of 
patterns appeared to shift with practice, Razran interpreted 
his results as contradicting the static laws of pattern organiza- 
tion implied in some of the Gestalt writings. 

Tonal patterning has been studied by Kleshchov (1932a- 
1936). He found, for example, that after conditioning to a 
musical fifth, generalization was greater to other pairs of tones 
when their ratio was that of a fifth rather than a third. This 
represents a form of equivalence included in von Ehrenfels’s 
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definition of a Gestalt-quality. Related observations have 
been reported by Razran (1938d). 

The component stimuli of a combination may be differen- 
tiated from the whole without loss of effectiveness by the 
whole. It is possible to condition an animal to make one re- 
sponse to a pattern of stimuli and another, qualitatively dif- 
ferent, response to the individual components (Beritov, 1932). 
The reference experiment for this type of differentiation em- 
ploys the procedure known as conditioned inhibition (see 
p. 126). A given stimulus is invariably reinforced when it is 
presented alone. When a second conditioned stimulus is pre- 
sented along with it, the combination is not reinforced. It is 
evident that conditioned inhibition is simply one arrangement 
for discrimination by the method of contrast. The contrast in 
the stimuli is produced by addition of a stimulus rather than 
by a change in some dimension of the original positive stim- 
ulus. An instance of conditioned inhibition in eyelid responses 
of human subjects has been reported by Shipley (1934b). A 
buzz provided the positive conditioned stimulus, followed by 
a striker to the cheek which produced a lid response. A flash 
of light served as the conditioned inhibitor; when both light 
and sound were present together, reinforcement was omitted. 
The buzzer alone did not lose its capacity to evoke a response, 
although responses to it were reduced when it was presented 
in combination with the light. It is, of course, possible to re- 
verse the usual practice of conditioned inhibition, and have 
the combination positive, the individual components both neg- 
ative. The important thing is that one pattern of stimuli may 
be differentiated from another, and the parts do not serve in 
the whole as they serve alone. 

Patterns of Successive Stimuli. The difference between 
the positive and the negative stimulus may be one of arrange- 
ment, rather than one of addition or subtraction. Thus geo- 
metrical figures alike except for their orientation, or sequences 
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of stimuli similar in component parts but dissimilar in order, 
may be discriminated. In some early experiments from Pav- 
lov’s laboratory four tones of vibration frequencies of 290, 
325, 370, and 413 d. v. were employed in different permuta- 
tions, only the order 1-2-3-4 being reinforced. There are 24 
permutations, such as 4-3-2-I, 2-1-3-4. According to Pavlov 
(1927, p. 146) all of the sequences were absolutely discrimi- 
nated from the one sequence which was reinforced. This was 
a pattern of ascending pitch. Differential response to it illus- 
trates a patterning or organization of perceptual response, as 
emphasized prominently by Gestalt psychology. Humphrey 
(1928b) showed how patterning in an arpeggio or melody 
could conceal a particular tone to which finger withdrawal had 
been conditioned. 

Generalization and the Definition of Similarity. In the 
experimental treatment of similarity it is possible to select 
stimuli arranged according to a logical or conventional dimen- 
sion of difference, and then to discover the amount of gen- 
eralization which occurs to stimuli along this dimension. The 
end result is a refining of the scale of similarity along this 
dimension for the organism under consideration. Confusion 
arises only if it is asserted that the organism is responding to 
similarity in the stimuli, rather than that it is responding to 
the stimuli as similar. The former statement requires a defini- 
tion of similarity independent of the organism under question, 
the latter makes no such demand. 

Response to similarity in the stimuli is based on the con- 
ception that stimuli are physically similar. The limitations of 
a physical definition of similarity are apparent in the study 
of the generalization of conditioned responses to tones of dif- 
ferent pitch. Humphreys (1939c) established a conditioned 
galvanic response to a tone of 1967 cycles and tested the mag- 
nitude of the generalized response to other tones. The re- 
sponse to a tone of 984 cycles, which is an octave of the re- 
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inforced tone, was reliably greater than that to a tone of 1000 
cycles, which is more similar to the reinforced tone in terms of 
physical frequency. 

There are a number of types of equivalence between stimuli 
(and the responses which they evoke) which cause the stimuli 
to be reacted to as similar, so that conditioned responses 
formed to one of the stimuli will be generalized to the other. 
Among these may be mentioned: partial identity (whereby 
stimulus A + B is similar to A + C); sensory similarity (so that 
two colors are more alike than a color and a tone); formal 
similarity (for example, two patterns of ascending pitch); 
affective similarity (equivalence mediated by emotionally 
flavored processes, as in the ‘dark-brown-taste’ or ‘blue Mon- 
day’); and mediated similarity (as in the equivalence of words 
of different sounds, but of common meanings). Of these rela- 
tionships, only the first (partial identity) may be conceived 
of as physical similarity. The second (sensory similarity) may 
be mediated by relatively primitive physiological processes, as 
represented in the classical experiments on generalization 
within a sensory modality. The others all depend upon rela- 
tively complex processes, many of which require previous 
learning in order that the stimuli involved may be responded 
to as similar. 

Learned mediation in conditioning experiments may be 
illustrated by an experiment in which physical equivalence is 
clearly out of the question. Children 7 and 8 years old, taught 
to give conditioned grasping responses to sound or to light, 
gave the responses when the words “sound” or “light” were 
spoken by the experimenter, or were presented as printed 
words on cards (Smolenskaya, 1934). It is evident that the 
similarity between a word and that for which it stands Ba 
similarity of reference which has been learned in the earlier 
experience of the child. Such similarity has to be acquired. 

Lundholm (1928, 1932) showed that it was possible to utilize 
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hallucinated stimuli in hypnosis as a basis for conditioning 
and extinction. Responses produced in the presence of hallu- 
cinated lights were given as conditioned responses to genuine 
lights. In his experiments the verbal suggestions of the experi- 
menter aroused processes which substituted for sensory stim- 
ulation. 

Mediation depends upon intermediate responses of some 
sort. This interpretation is supported by the fact that long- 
trace conditioned responses tend to be generalized widely. 
According to Pavlov, they exhibit “a permanent and universal 
generalization, involving all the analyzers” (1927, p. 113). 
Accepting the interpretation of trace responses as conditioned 
to the subject’s own responses (p. 171, Chapter 7) generaliza- 
tion of these trace responses may be understood in terms of 


the similarity of the postural adjustments made to a variety 
of stimuli. 


Summary 


When a conditioned response has been developed by re- 
peated reinforcement of one stimulus, another similar stimulus 
may elicit the (qualitatively) same conditioned response with- 
out reinforcement. If stimuli are scaled along a single psycho- 
physical dimension (such as pitch or loudness of tones), the 
degree of generalization may be expressed as a gradient, gen- 
eralization decreasing as the stimuli become less similar. With 
repeated reinforcement, generalization becomes at first more 
widespread, and then somewhat more restricted, provided 
generalization is expressed as a ratio of the generalized re- 
sponse to the conditioned response. In terms of absolute 
magnitude, the conditioned response generalized to neighbor- 
ing stimuli tends to increase in size as reinforcement of the 
original conditioned stimulus continues. Generalization may 
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be reduced by the method of contrasts, in which one of a pair 
of stimuli is always reinforced, the other not reinforced. Dis- 
crimination then results, provided the stimuli are sufficiently 
different. The method may be used for studying the discrim- 
inatory abilities of animals, both in respect to differentiation 
along a single dimension, and in respect to the patterning of 
stimuli either in simultaneous patterns or in serial patterns. 
When compound stimuli are reinforced, the effectiveness of 
separate components for eliciting the conditioned response 
must be tested empirically. The effectiveness varies both with 
the nature of the component stimuli and with the stage of 
training, so that a simple redintegrative principle is not suffi- 
cient to account for the data. 

Generalization makes possible an empirical approach to 
the problem of similarity. Among the types of similarity which 
have been discussed briefly are partial identity, sensory simi- 
larity, formal similarity, affective similarity, and mediated 
similarity. 


Notes 


Choice-Reaction Discrimina- 


tion 


Gradients of Generalization 


The form of the gradient of gen- 


eralization is important in the 
attempts to derive phenomena of 
discrimination from generalization. 
There is no reason to believe that 
one form of the gradient will be 
found universal. For discussions of 
the problem cf. Hull (1939b), Raz- 
Tan (1938c), and Spence (1939). 
Loucks’s (1933, 1937b) criticisms of 
the irradiation studies are directed 
against temporal aspects of gen- 


eralization, not against the spatial 
aspects. 


Choice-reaction discrimination 
cannot be accounted for completely 
through reinforcement and extinc- 
tion of the alternative responses, 
because there are many other com- 
plications in the arrangements, such 
as the introduction of delays, pun- 
ishments and rewards before choice, 
at the moment of choice, and after 
incorrect choices (Muenzinger and 
associates, 1935—-1937)- Shock at the 
moment of choice may delay learn- 
ing through general disorientation 
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with respect to the stimuli, leading 
to position habits (Muenzinger and 
A. Wood, 1935; Fairlie, 1937), just 
as delay prior to choice may im- 
prove learning through making the 
animal more attentive to the stim- 
uli to be discriminated (Muenzinger 
and Fletcher, 1936; Honzik and 
Tolman, 1938). The theoretical 
alternatives to conditioning explana- 
tions of behavior at a choice-point 
are presented by Tolman (1938a). 


Mechanism of Discrimination 
Learning 


The treatment of discrimination 
learning proposed by Spence (1936) 
initiated a controversy which carries 
on earlier disputes between expo- 
nents of association theories and 
the competing theories of Gestalt 
and sign-learning. This continues at 
the time of writing. 

Krechevsky’s studies of pre- 
solution hypotheses are presented in 
a series of papers (1932a, 1932b, 
1933a, 1933b). Only a detailed 
analysis can factor out the relative 
significance of systematic solution 
(e.g., with ‘insight,’ ‘hypotheses’) 
and solution determined as an alge- 
braic consequence of reinforcements 
and non-reinforcements. Papers 
bearing directly on this aspect of 
the problem of discrimination are: 
Haire (1939a, 1939b), Hull (1939b), 
Humphreys (1939a, 1939c, 1940c), 
Krechevsky (1937, 1938), T. L. 
McCulloch (1939, 1939b), Spence 
(1936, 1938). The interpretation of 
sudden solution in terms of insight 
has been proposed by many writers, 
of whom Köhler (1925) and F. T. 
Perkins and Wheeler (1930) may 
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be taken as representative. The 
problems of relative and absolute 
discrimination (i.e., transposition) 
have received treatment recently by 
Gulliksen and D. L. Wolfle (1938a, 
1938b), Hull (1939b), Jackson and 
Dominguez (1939), Jackson, Stonex, 
Lane and Dominguez (1938), F. 
N. Jones (1939a, 1939b), Korotkin 


(1938), Lashley (1938), Maier 
(1939a), Razran (1938c, 1939e), 
Spence (1937b, 1939), Tolman 


(1939). The theory proposed by 
Brunswik (1939) goes beyond the 
others in embracing many addi- 
tional facts of perception, such as 
those reported by Fieandt (1938), 
who relates perceptual learning to 
the problem of sensory condition- 
ing. 

Two mathematical treatments of 
discrimination are concerned with 
other problems—Householder (1939) 
with a neural mechanism, Hilgard 
(1938) with the relative influence 
of set and voluntary factors in con- 
ditioned discrimination in man. 


Threshold Measurement 


Classical conditioning has been 
widely used as a method for deter- 
mining sensory abilities in animals 
and infants. The early studies on 
dogs from Pavlov’s laboratory were 
reviewed by Yerkes and Morgulis 
(1909); later studies were sum- 
marized by Razran and Warden 
(1929). Razran has also reviewed 
the work on animals other than 
dogs (1933a) and on children 
(1933b). Many inaccessible studies 
are reported by Beritov (1927). 

The most extensive recent work 
has been on audition. As illustrative 
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of studies reporting measurements 
by conditioning and closely related 
techniques are: Ades, Mettler, and 
Culler (1937); Andreyev (1934a); 
Anrep (1920); Britt (1935); Brog- 
den and Culler (1937); Brogden, 
Girden, Mettler, and Culler (1936); 
Bull (1928); Culler (1936); Cul- 
ler, Finch and Girden (1935); Cul- 
ler, Finch, Girden, and Brogden 
(1935); Culler, Willmann, and 
Mettler (1937); H. Davis, Derby- 
shire, Kemp, Lurie, and Upton 
(1935); Dworkin (1934b, 1935a, 
1935b); Dworkin, Seymour, and 
Sutherland (1934); Elder (1934, 
1935); Engelmann (1928); Girden 
(1935, 1938, 1939); Henry (1938); 
Herington and Gundlach (1933); 
Horton (1933, 1934, 1935); Kasat- 
kin and Levikova (1935); Kemp 
(1936); Kleshchoy (1932a-1936); 
Muenzinger and Gentry (1931); 
Neet (1936); Ten Cate (1934, 


1934b); Thuma (1932); Upton 
(1929); Wendt (1934); Wever 
(1930). 


_Examples of studies dealing with 
visual sensitivity are those of Bogo- 
slovski (1937, 1938), R. H. Brown 
(1936a, 1936b, 1937), Bull (1928, 
1930, 1935), Frolov (1924, 1925a), 
Kennedy and K. U. Smith (1935), 
D. G. Marquis (1934), Newhall and 
R. R. Sears (1933), K. U. Smith 
(1934, 1935), Spence (1934), Ten 
Cate (1934a). 

Olfactory and trigeminal re- 
sponses have been studied by W. F. 
Allen (1937, 1938), temperature 
sensitivity by Bull (1936), static 
and kinetic reactions by Oppen- 
heimer and Spiegel (1937), vibra- 
tory sense by Sutherland and Dwor- 
kin (1932). 


Studies of conditioning to sub- 
liminal stimuli have been carried 
out by Cason and Katcher (1933), 
Newhall and R. R. Sears (1933), 
Silverman and Baker (1935), Neet 
(1936), Baker (1938). 

Studies of sensory processes con- 
cerned primarily with the neural 
mechanisms involved are reported 
in Chapter 13. 

À precaution always to be kept in 
mind in studies of capacity is that 
the indicator chosen by the experi- 
menter may not be a sensitive one 
from which to infer the organism’s 
potential discriminations by other 
methods. Discriminatory condition- 
ing, for example, does not always 
occur, even when the to-be-dis- 
criminated stimuli are well above 
perceptual thresholds as measured 
in other ways (Cole, 1939; Hilgard, 
A. A. Campbell, and W. N. Sears, 
1937; Hilgard, R. K. Campbell, and 
W. N. Sears, 1938). 


Similarity and Difference as 
Experimental Problems 


Discriminatory reactions have 
taken on new importance in psy- 
chological theory with the increas- 
ing recognition of the operational 
nature of the concepts used in lab- 
oratory investigations. Thus Boring 
(1933, p- 234) says: “Consciousness 
is discriminative, and discrimination 
is the symptom of consciousness.” 

Hull (1939b) considers stimulus 
equivalence to be dependent on 
three fairly distinct mechanisms: 
(1) partial physical identity, (2) 
primary or physiological generaliza- 
tion (irradiation), and (3) second- 
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ary generalization. Under secondary 
generalization he would include the 
mediated varieties of similarity dis- 
cussed in the text. 

Studies of the transfer effect of 
learning to respond to one stimulus 
on the acquisition of the same re- 
sponse to a second stimulus present- 
ed together with the first have been 
carried out by Winslow (1938, 1939) 
and Winslow, Kantor and Warden 
(1938). Successive combinations of 
this type required fewer trials than 
the preceding ones. The relation of 
this type of training to higher order 
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conditioning, as studied by Finch 
and Culler (1934), was pointed out 
by the authors. 

The Russian studies on general- 
ization and discrimination of pat- 
terns of simultaneous and succes- 
sive stimuli have been little accessi- 
ble to English readers. They have 
recently been made available 
through a comprehensive review by 
Razran (1939a). The most exten- 
sive program has been carried out 
in Beritov’s laboratory, with loco- 
motor situations using dogs and 
rabbits. 


Chapter 9 
SERIAL LEARNING 


HE BASIC principles and experimental laws of condition- 
ake ing which have been outlined in the last five chapters 
were derived from relatively simple and uncomplicated ex- 
periments which had been designed expressly to demonstrate 
such principles as clearly as possible. One of the chief inter- 
ests in conditioning among psychologists today, however, lies 
in the systematic relation of conditioning to general problems 
of learning. What light does the study of conditioning throw 
on the learning of complex skills such as typewriting, card 
sorting, and maze running, or the solution of complex prob- 
lems involving reasoning and insight? This question has been 
discussed and answered in various ways, as indicated in Chap- 
ter 1. The most carefully conceived and elaborated theory of 
learning based primarily on principles discovered in condi- 
tioning experiments is that of Hull. He rejects as superficial 
and misleading the simple conceptions of conditioning as a 
single law of association by contiguity, a building-block unit 
of which habits are composed, or an analogue of all learning 
situations. Conditioning is, rather, a source of behavioral prin- 
ciples which can be utilized as postulates for the deduction of 
Statements about more complex learning. 

The postulates of any theoretical system, whatever their 
sources, are considered logically as pure assumptions. The 
theorems or hypotheses which are derived by deduction from 
the postulates are subjected to experimental test, and the 
postulates are retained in the theory so long as the deductions 
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check with fact. With this freedom in the choice of postulates, 
the theorist about learning will naturally be guided to select 
those relationships which appear most plausible and compre- 
hensive. Most of the postulates in Hull’s system are essentially 
the same as the experimental laws of conditioning described 
in the previous chapters. For his purposes, they become as- 
sumptions; they are accepted as stated and the problem be- 
comes that of understanding some of the more complicated 
phenomena of learning in terms of them. 

Although Hull’s theoretical formulations represent an at- 
tempt to comprehend complex learning situations in terms of 
conditioning principles, there is no implication that habits are 
merely compounds of conditioned response units. He has 
expressly stated this position as follows: 


In order to correct a frequent misunderstanding, due pre- 
sumably to the wide dissemination of the views of J. B. Wat- 
son, the writer wishes to make it quite clear that neither here 
nor in any previous publications has he assumed that the more 
complex forms of behavior are synthesized from reflexes 
which play the rôle of building blocks. This may or may not 
be true. His working hypothesis is, rather, that the principles 
of action discovered in conditioned reaction experiments are 
also operative in the higher behavioral processes. The com- 
pound adjective in the expression, ‘conditioned-reflex prin- 
ciples,’ accordingly refers to the locus of discovery of the 


principles rather than to their locus of operation. (Hull, 19350, 
Dp. 227n:) 


It is not necessary, according to Hull, to analyze complex 
behavior into its component conditioned responses. It is only 
hecessary to base hypotheses regarding more complicated 
learning on conditioning principles, and to test these hypo- 
theses by experiment. If success is achieved, common relation- 
ships may be inferred within conditioning and other learning. 
These relationships may themselves be in need of explana- 
tion, but it is evident that even before they are explained a 
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genuine scientific advance will be achieved if data superficially 
very unlike are harmonized around a few common principles 
of action. 

The extension of conditioning principles has been most care- 
fully worked out in two rather limited fields of learning. In 
this chapter an account of serial learning will be presented, 
and in the following chapter, problem solving behavior will 
be considered. 


Characteristics of Serial Learning 


In many experiments the learned activity is more than a 
single response such as an eyelid closure or a finger with- 
drawal. This is particularly true of instrumental training in 
which the reward is secured only after a series of responses 
have been carried out. A trained dog will respond to the com- 
mand “Fetch it” by running out into the field, picking up a 
stick, returning, and laying the stick at the trainer’s feet. Even 
the simplest instrumental responses involve a series of move- 
ments. The rat in the Skinner apparatus secures food by tak- 
ing successive steps to approach the region of the lever, lifting 
its forelegs from the floor, pressing downward on the lever, 
lowering its body to the food cup, lowering its head, and open- 
ing its mouth. In a single straight alley, the locomotion of the 
rat from one end to the other to secure food may be consid- 
ered a serial response by analyzing the speed of running sep- 
arately for the successive segments of the total path. 

When an instrumental act is considered as a single response, 
the principles of conditioning may be directly applied. For 
example, the maze reaction, considered as a unit, will be more 
rapidly acquired if the reward is large and the animal is 
hungry; it will be more slowly learned if the reward is de- 
layed. Gradual disappearance of the reaction will follow trials 
on which the reward is omitted, and spontaneous recovery 
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from this extinction will occur during an interval of no train- 
ing. There are many facts about maze learning, however, 
which cannot be encompassed by this direct application of 
conditioning principles to the reaction as a unit. There is no 
possibility of explaining why the rat takes a shorter rather 
than a longer route through the maze, why some blind alleys 
are eliminated sooner than others, why mazes of different 
pattern vary in difficulty. The answers to such questions can 
be obtained only if the total maze reaction is analyzed in terms 
of its component part reactions. An adequate account of the 
serial organization of the habit as a whole calls for a consid- 
eration of the stimuli evoking the successive responses, the 
reinforcement of the separate responses, and the condensation 
or short-circuiting of serial acts. 

Stimuli for Serial Reactions. The stimuli which evoke 
the successive responses in a serial act must occur in a fixed 
and unvarying sequence. There are two possible ways in which 
this order is guaranteed. The first requires the presentation 
of a series of environmental stimuli, to each of which a re- 
sponse is made. The serial nature of the response thus depends 
upon the regularity of the external stimuli. If a rat has been 
trained in a discrimination problem always to go to a lighted 
doorway in preference to a dark one, its route through a mul- 
tiple discrimination apparatus, which presents 10 such choices 
in succession, will be determined by the visual stimuli. After 
many trials in this situation, with the lights unchanged, the 
Tat may take the same route even when no visual cues are 
present. The stimuli for the successive responses are NOW 
presumably internal. Each act itself provides proprioceptive 
stimuli for initiating the one next in the series. This second 
way of securing a fixed order of stimuli is customarily called 
chained conditioning. Pavlov and subsequent writers have 
placed emphasis on the interpretation of both instincts and 
habits as chained responses. Holt (1931) has pointed out the 
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importance of chaining in such circular reflexes as the grasp 
of the new-born infant. Stimulation of the palm of the hand 
evokes grasping, and the closing of the fist prolongs and in- 
creases the stimulation of the palm, thereby increasing the 
strength of the grasp. Another illustration of chained reflexes 
is provided in the act of swallowing, in which each component 
movement propels the food farther along the alimentary tract 
and thus provides the stimulus for the next successive move- 
ment. 

Learned serial habits presumably involve chained responses 
similar to reflex chains, except that the connections between 
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Ficure 26 
THE INTEGRATION OF SERIAL LEARNING 


The reactions Ri, Ra, Rs, Ru, Rs to the external stimuli Si, Sz, Ss, Ss, Ss 
become integrated as a series by the proprioceptive stimuli s:, Sz, ss, ss from 
the muscles used in the reactions. Original responses are represented by 
solid arrows, learned responses by broken ones. After Hull (1930a). 


the stimuli and the responses are conditioned. In view of the 
importance of this mechanism it is strange that so few experi- 
mental demonstrations of the chaining of conditioned re- 
sponses have been made. It is evident from several studies 
that reaction-produced stimuli may function as conditioned 
stimuli for other responses, and it is therefore altogether 
plausible that chaining operates in serial learning. An analysis 
of the process in conditioning terms by Hull (1930a) is pre- 
sented in Figure 26. S,, S, etc., represent the successive ex- 
ternal stimuli provided by the environment, each evoking in 
the organism a separate response, R,, R,, etc. The sequence of 
responses is entirely dependent on the serial order of the ex- 
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ternal stimuli. But each response produces a characteristic 
proprioceptive stimulus complex such as sı, so. By the prin- 
ciple of substitution, the internal stimulus sı, which in repeti- 
tions of the total situation always occurs at the same time that 
S2 evokes Rg, will come to evoke that response in the absence 
of the external stimulus. The dotted arrows in the diagram 
represent the newly acquired conditioned responses. As a 
result, if the sequence of external stimuli is interrupted after 
Sı occurs, the serial responses may nevertheless be called 
forth. 

In most situations the integration of a behavior series de- 
pends both upon the chaining mechanism and upon the se- 
quences of stimuli in the environment. Thus the restricted 
pathways of a maze confront the organism with a systematic 
sequence of external stimuli, if it but runs through the maze. 
Similarly, a revolving memory drum will expose new items 
to the learner if he but waits and watches. Whatever integra- 
tion he may achieve because of the conditioning of reaction- 
produced internal stimuli, the learned serial behavior often 
requires the support of the orderly series of environmental 
events. An individual who has learned to follow a finger-maze 
correctly may not be able to draw its pattern on paper without 
the support of the stimulus cues furnished by the maze itself. 

Reinforcement of Serial Reactions. The component re- 
sponses of a serial reaction are usually not separately and in- 
dependently reinforced. Reward is not given following each 
Segment of a maze, but only at the termination of the sequence. 


The important consequences of this fact have been pointed 
out by Hull in his goal gradient hypothesis: 


The mechanism . . - depended upon as an explanatory 
and integrating principle is that the goal reaction gets condi- 
tioned the most strongly to the stimuli preceding it, and the 
other reactions of the behavior sequence get conditioned to 
their stimuli progressively weaker as they are more remote 
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(in time or space) from the goal reaction. (Hull, 1932, pp. 
25-26.) 


This principle, as suggested in Chapter 7 (p. 169), may be 
considered an extension to the serial learning situation of the 
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FIGURE 27 
Locomotion Grapient oF Rat in 40-Foor RUNWAY 


_ Composite graphs from 14 animals showing the time consumed in travers- 
ing each of the eight 5-foot sections at the two extremes of training and a 
subsequent restoration of the gradient following non-reward (i.e., absence 
of food at the end of the alley). Although the gradient has virtually dis- 
appeared by the end of training, non-reward restores it in its earlier form. 
(Hull, 1934b, p. 404.) 
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gradient of reward. Let Figure 26 represent the five successive 
segments of a straight single-alley maze which a rat traverses 
to secure food reward at the end. The final act R, will become 
conditioned to the stimulus complex S, +s, more strongly than 
any of the previous acts become conditioned to their stimuli 
because it is closer to the terminal reinforcement. Responses 
4, 3, 2, and 1 will be progressively less strong. As applied to 
running speed, Hull’s experimental test of this principle is 
illustrated in Figure 27. 

Condensation of Serial Reactions. It is a common ob- 
servation that some of the component responses in a serial 
reaction tend to drop out during continued practice. This 
short-circuiting tendency will result in more efficient perform- 
ance when the eliminated components are ‘errors’ or un- 
necessary movements. The typist gains speed by eliminating 
the visual locating of the proper keys; the child learns to 
count without touching his fingers. In other instances the 
short-circuiting tendency results in less efficient performance, 
as when a person steps on the starter in his automobile be- 
fore he has turned on the ignition, or begins to run before he 
has securely caught the football thrown to him. But no matter 
what the outcome, the tendency toward condensation of serial 
reactions is a familiar one. 

An explanation of the mechanism of condensation in terms 
of conditioning principles has been offered by Hull (19302). 
He points out that in addition to the successive external and 
internal stimuli controlling a serial reaction, there is a per- 
sisting stimulus which remains fairly constant while the suc- 
ceeding responses occur. This is represented by the symbol 
Sp in Figure 28. It may be thought of concretely as a con- 
unuous green light, or the sight of the memory drum in which 
the Separate items are presented, or the stimuli arising from 
continuous knitting of the brows, or (more typically, in animal 
experimentation) the stimuli arising from a continuous drive 


SERIAL LEARNING 215 


condition such as hunger. From Figure 28 it is apparent that 
Sp will be conditioned to each of the responses, but according 
to the goal gradient principle, will be most strongly condi- 
tioned to R,, somewhat less to R,, and so on. If at any time 
during learning the tendency of Sp to evoke R, should be 
stronger, for example, than the tendencies of S,, s, and Sp 
together to evoke R,, the final response will occur and seg- 
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FIGURE 28 
THE INTEGRATIVE FUNCTION OF AN ENDURING STIMULUS 


To the stimuli and responses of Figure 26 have been added the enduring 
stimulus Sp, conditioned to each of the responses in the series, as shown by 
the broken arrows (Hull, 1930a, 1931). All of the conditioning is influenced 
by the reinforcement based on the goal situation at the end. 


ments R,, R;, and R, will be short-circuited. The premature 
intrusion of a response is frequently designated as an antici- 
patory response. The anticipatory characteristic of simple 
conditioned responses has been pointed out before (p. 162). 
The mechanism may be similar to that described here, for the 
conditioned stimulus (or some consequence of it) persists 
from its onset until the reinforcement occurs. Complete short- 
circuiting such as that just described is not adaptive if the 
anticipatory response interrupts progress to the goal. There 
are some anticipatory reactions, however, which are not in 
conflict with progress toward the goal. Among these, for ex- 
ample, are chewing movements and salivation preparatory to 
eating at the end of the maze. Such fractional anticipatory 
goal reactions become very important in the later extensions 
of Hull’s theory. 

Following this brief survey of the characteristics of serial 
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reactions, we will turn to a consideration of the two most ex- 
tensively studied examples of serial learning—the maze, and 
rote verbal learning. 


Maze Learning 


The value of the maze for the analytical study of serial 
learning lies in the ready identification of the component units 
of the series. The pathway down which the animal runs con- 
tains a number of bifurcations, one path ending as a blind 
alley, the other leading forward toward the reward at the 
end. At each such point the animal makes a choice-reaction 
discrimination. The stimulus basis of the discrimination is a 
complex one, comprising external as well as proprioceptive 
stimuli. The chief problem in the study of maze learning is 
concerned with the explanation of successful choice-point re- 
actions. With reward inevitably following either correct or 
incorrect choices, how do the blind alleys ever become elimi- 
nated? 

Although other writers have discussed the maze in relation 
to conditioning, the most thorough attempt to account for 
the details of maze learning in terms of conditioning principles 
is that of Hull (1932, 1934a). The goal-gradient principle is 
made one basis of blind alley elimination. Although either a 
Correct or an incorrect response will be reinforced by the 
reward at the end, the incorrect response receives less re- 
inforcement because it is more remote from the goal (Figure 
29). The path which represents a shorter distance to the goal 
will come to be chosen in preference to one which represents 
a greater distance. 

The rapidity of learning the correct response at any choice- 
point depends upon the difference between the strengths of 
reinforcement of the two alternative responses. If the differ- 
ence is small many repetitions will be necessary before the 
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Short path 


Long path 





FIGURE 29 


EXPLANATION OF THE PREFERENCE FOR THE SHORTER OF ALTERNATIVE PATHS 
Accorpinc to Hutr’s Goat GRADIENT HYPOTHESIS 


In the upper diagram a maze is shown in which the short path consists of 
5 units of length from start to goal, the long path, of 7 units. The stimuli 
associated with each unit are simplified as a series Sa to Sa and Sı to Sa 
for the two paths. The goal gradient hypothesis assumes that sequences 
of stimuli and responses as represented in the lower portion of the figure 
will be strengthened in proportion to their distance from the goal. If, then, 
there have been equal runs by way of the two paths in the past, when the 
animal is now confronted with the two stimuli Sa and Sı, the choice will 
be made on the basis of the tendency of Sa to evoke Ra as compared with the 
tendency of S, to evoke Ri. It is evident that the sequence S:—Ra will be 
the more strongly conditioned because it lies nearer the goal. Therefore 
the short path will be entered. Stimulus components furnished by the drive 
and the movements of the organism have been omitted for purposes of 


simplicity in exposition. They have been included by Hull (1932). 


error response is completely eliminated. If the difference is 
large, learning will be more rapid. Experimental results 
(Yoshioka, 1929) demonstrate that it is the relative rather 
than the absolute difference between the two reinforcement 
strengths which determines the speed of learning. From this 
it follows that long blind alleys will tend to be eliminated more 
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Ficure 30 


EXPLANATION OF THE More RAPID ELIMINATION or BLINDS NEAR THE GOAL 
Accorpinc TO Huuu’s Goat GRADIENT HYPOTHESIS 


By determining the difficulty of eliminating blind 6b’ in Maze A and 
blind aa’ in Maze B, a comparison may be secured between a blind near 
the goal and one more remote. It is assumed that the choice-point behavior 
is determined by the relative lengths of paths with a blind compared with 
the length without the blind. Thus, at choice-point b of Maze A, the tend- 
ency to take the true path as compared with the blind may be expressed 
as the ratio of their lengths: 

For choice-point b, Maze A: . 

(Path with blind) _ bb’ + b’b+ bg 


So 5 530 
(Path without blind) bg I 
Similarly, for choice-point a, Maze B: 
(Path with blind) _aa' +a'a+ab + bg E 
(Path without blind) ab + bg ToT 


Since the relative difference of true path and path with blind is greater in 
Maze A than in Maze B, blind bb” will be more readily eliminated than 
blind aa’. This is what the deduction set out to prove. It is not assumed that 


this is the only principle applicable, or that the comparison could be made 
experimentally in mazes as simple as those diagrammed. 
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readily than short ones, since a long blind results in a greater 
relative lengthening of the path than a short one. For similar 
reasons, a blind alley near the goal should be eliminated more 
readily than one near the entrance. The argument is illustrated 
in Figure 30. There are sufficient confirmations of these pre- 
dictions in certain types of mazes to have excited a great deal 
of interest in Hull’s analysis. It is evident that predictions of 
this kind go beyond the mere assertion that maze-behavior is 
conditioned. A single analogy from conditioning would , not 
suffice to make the predictions. 

Experimental tests of the goal gradient principle in maze 
situations have given somewhat ambiguous results, because, 
whether or not the goal gradient principle is correct, many 
other factors are also present. Three of these additional fac- 
tors are quite important. Centrifugal swing (Schneirla, 1929; 
Ballachey and Buel, 1934a) is a matter of the inertia of 
running, which tends to favor either continuing in a straight 
path, or running off at a tangent after a turn. This is a matter 
of performance, rather than of learning, and determines be- 
havior in the very first run through the maze, before there 
has been any chance for learning to occur. This factor of the 
general inertia of running is not as effective in linear mazes, 
which are therefore preferable for use in testing principles 
such as the goal gradient. A second factor significant in maze 
learning is the tendency to anticipate later turns, particularly 
the final turn leading to the goal. A third factor is the tendency 
of the animal to orient in the general direction of the food-box. 
In a linear maze with food on the right of the center, more 
errors are made in alleys on the right than in alleys on the 
left. This may be a turning habit based on the anticipation of 
the last unit of the maze instead of an orientation to the food- 
box (F. N. Jones and Taylor, 1938). Figure 31 illustrates a 
linear maze used by Spence and Shipley (1934), in which food 
Orientation, goal gradient, and anticipation may all be effec- 
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FIGURE 31 


PLES OF 
Maze Usep BY Spence AND SHIPLEY (1934) In Testne PRINCI 
RELATIVE DIFFICULTY or Buinps 


The dotted path shows a modification of the maze introduced by F. N. Jones 
and 


= iment 
Taylor (1938) placing the food-box on the left. In their experime 
entrances were 


5 . 2 5 hat 
also more frequent in the right-hand blinds, showing t 
anticipation of the right turn 
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1S more important in this maze than orientat 
in the direction of the food-box. 


SERIAL LEARNING 221 


tive. Indications within the data suggest that at different 
stages of the learning different factors are important. Thus 
within the first five trials it was found that the errors became 
increasingly more frequent in the goal-pointing rather than in 
the non-goal-pointing blinds. The order of elimination of 
blinds was according to the goal gradient, most errors being 
made in blinds remote from the goal. Nearer to mastery, how- 
ever, the order was reversed, more errors being made in the 
blind just prior to the goal-turn. This suggests the prominence 
at this stage of the anticipatory tendency. 

Alleys pointing away from the goal are believed by Hull to 
be eliminated largely on the basis of the competing alternative 
tendencies which are more strongly reinforced according to 
the goal gradient, while the major factor in the elimination of 
blinds pointing toward the goal is believed to be inhibitory 
in nature, based on the extinction of the tendency to enter 
goal-pointing alleys (Hull, 19342). If maze behavior is in 
accordance with these principles, important deductions follow 
with regard to the different effects of spontaneous recovery, of 
spaced and massed practice, and of the injection of drugs on 
behavior in the two types of alleys. By combining the goal 
gradient hypothesis with other conditioning principles such as 
extinction, external inhibition, disinhibition, generalization, 
and spontaneous recovery, Hull has proposed a number of 
specific deductions. The growing literature on maze behavior 
contains many studies related to these deductions. The data 
indicate that his account, with all its detail, does not cover all 
the complexities of maze behavior. Nevertheless, the papers 
are models of the possibility of deducing phenomena which 
are not directly analogous to conditioning experiments. 


Serial Verbal Learning 


Rote memorization of a list of words or nonsense syllables 
18 a serial task that has been the subject of extensive study. In 
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the customary training procedure, the successive items are 
exposed singly by a memory drum apparatus with an interval 
of 2 or 3 seconds between items. The first item serves as the 
stimulus for the response of pronouncing the second; the ap- 
pearance of the second item indicates to the subject that his 
response was right or wrong, and in turn serves as the stimulus 
for pronouncing the third item. The serial character of the task 
is guaranteed at first by the fixed order of item presentation, 
and after mastery, each response by the subject evokes the 
next in order without any stimulus cues from the environment. 
The reinforcement for the acquisition of new associations in 
this situation is of a complex derived type involving prompting 
and knowledge of results. It is important to note that rein- 
forcement occurs after each item in the series and not just at 
the end of the series. In this respect serial verbal learning 
differs from animal maze learning, and the goal gradient 
hypothesis would not be expected to apply. 

While relationships between memorization and condition- 
ing had been suggested earlier, Lepley (1932) made the first 
attempt to apply conditioning principles deductively to the 
phenomena of rote learning. He argued that each item in the 
Series serves as the cue or conditioned stimulus for eliciting 
later syllables as conditioned responses. The tendency to re- 
spond with the immediately following item may be thought 
of as a simultaneous conditioned response, while the tendencies 
to respond with items farther along in the list (remote associ- 
ations) may be considered as trace conditioned responses. The 
characteristic anticipatory tendency of trace responses results 
in the erroneous intrusion of items from later portions of the 
saries: The suppression of these anticipatory intrusions 1s 
said to be an inhibitory process with properties which resemble 
internal inhibition. From this analysis, Lepley deduced several 
of the characteristics of rote learning, most important of which 
is the decelerated form of the curve of retention. Initial for- 
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getting is rapid because the remote tendencies which were 
inhibited recover rapidly and appear in the retention tests as 
erroneous responses. Since these remote tendencies are not as 
firmly fixed as the direct simultaneous tendencies, the curve 
flattens out as time passes, and residual memory persists for 
a long time. 

Hull (1935a) expanded the Lepley hypothesis on the as- 





FIGURE 32 


An EXPLANATION IN TERMS oF ĪNHIBITION oF DELAY FoR THE DIFFICULTY 
IN MemorizaTION AT THE MIDDLE oF A SERIES OF NoNSENSE-SYLLABLES 


_ The figure, which is taken from Hull (19352), shows remote associations 
in the forward direction by curved lines. Backward associations are neglected 
for purposes of simplicity. The amount by which intervening items are 
interfered with (‘inhibited’) may be determined by counting the number 
of lines spanning each item. The numbers are presented below each syllable. 
It is evident that the ends of the list will be favored in memorization, while 
items toward the middle will be progressively more difficult to learn. 


sumption that, when remote associations are formed, inter- 
vening items are inhibited in accordance with the principle of 
inhibition of delay. He accounts in this manner for the fact 
that the middle items of the list are harder to memorize than 
the items at either end. How an analogy to the inhibition of 
delay may be used to account for the relative difficulty of items 
within a memorized series is illustrated in Figure 32. Every 
associative tendency which is shown to span an item is as- 
sumed to depress that item. By counting the number of such 
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connections passing over an item, the amount by which it is 
inhibited may be determined. 

The deductions which followed from Hull’s analysis led to 
several strikingly satisfactory confirmations. Many of the de- 
ductions were known to conform to facts already established 
about memory, for example, that there # greater difficulty at 
the middle than at either end of the series memorized, and that 
spaced practice is advantageous over massed practice. In 
addition, several novel phenomena were predicted and later 
confirmed in the laboratory. Among them was the theorem 
that the middle of the list should be recalled more satisfac- 
torily after the lapse of a short time, yielding an initial rise in 
the curve of retention. This follows from the assumption that 
inhibitory tendencies disappear more rapidly with time than 
excitatory ones. The predicted phenomena were demonstrated 
conclusively by Ward (1937), who also confirmed some pre- 
dictions with respect to the latent time of the responses to 
different items in the series. Hovland (1939) has shown that 
in paired associates learning in which serial character is de- 
stroyed by presenting pairs in altered random order on suc- 
cessive trials, the inhibitory tendencies within the series are 
less than those in serial anticipation learning so that there is 
found neither the initial rise in the curve of retention nor 
the advantage of spaced practice. One of Hull’s deductions, 
having to do with the effects of caffeine, was not completely 
confirmed (Hull, 1935b). The extent of agreement between 
predictions and findings has been remarkably satisfactory, 
and an occasional failure serves to indicate the direction which 
modification of the theory must take. In a more recent publi- 
cation, Hull and his collaborators (Hull, Hovland, Ross, M. 
Hall, D. T. Perkins, and F. B. Fitch, 1940), have presented a 
closely argued extension of the theory. More than 50 theorems 


are stated, and the proofs of their derivation are given in 
mathematical as well as in verbal form. 
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Because of the success achieved in the deductive account of 
serial verbal learning, there may be a tendency to overlook 
some of the difficulties in the analogies which have been used. 
The basic analogy in Hull’s (1935a) theory is that of the 
inhibition of delay. It is known experimentally that trace con- 
ditioned responses are very slowly established. The greater 
difficulty in the middle of the list, however, is found after the 
first time through the list, and is in fact more pronounced early 
than late in memorization (Robinson and M. A. Brown, 1926). 
This is quite unlike the establishment of a delay. In the second 
place, the tendency of delayed responses to occur prior to the 
reinforcement is very great. It is more likely that the inhibition 
observed in serial verbal learning represents an interference 
between the correct tendency and the tendency to respond 
with the remote association as a simultaneous response, rather 
than a true inhibition of delay. For example, when free as- 
sociation is permitted to the separate items of a memorized list 
presented in random order following memorization, the laten- 
cies of remote associations do not correspond at all to the 
actual time intervals between the items in the memorization 
(McGeoch, 1936). In the third place, the effect of a time 
interval after conditioning, which results in rapid recovery 
from inhibition, leads not only to a disappearance of the 
inhibition of delay, but to an exaggeration of the tendency to 
give trace responses at shorter latencies (Rodnick, 19372). 
This recurrence of anticipatory intrusions was part of the 
original Lepley (1932) hypothesis and accounted for rapid 
forgetting. Recovery from inhibition cannot be used to ac- 
count both for reminiscence through reduced inhibition (Hull, 
1935a) and for forgetting through increased interference 
(Lepley, 1932) without careful quantitative investigation of 
what is actually occurring. Some important alternatives to 
Hull’s analysis have been suggested by E. J. Gibson (1938) 
and by Hovland (1940) based on principles of generaliza- 
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tion and interference. These are unfortunately not yet avail- 
able in complete enough form for exposition. Their very ex- 
istence, however, indicates that Hull’s analysis must not be 
accepted as final, for alternative sets of postulates may be 
found to account equally well for the facts which Hull ex- 
plains. 

It is to be noted that the logic of the deductive approach to 
serial learning does not require the adoption of conditioning 
as a starting point. Associative strength may be defined em- 
pirically by the responses in free association experiments such 
as those of McGeoch (1936) or Raskin and Cook (1937), just 
as Spence (1936) in his deduction of discriminatory learning, 
defined his units of reinforcement by responses within a simple 
discrimination. The point at which analogies with condition- 
ing are most strained is in the identification of the unitary re- 
sponses which are said to be reinforced or extinguished. This 
difficulty is circumvented by accepting empirical findings at 
the point where the analogy with conditioning is the hardest 
to apply. 
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Notes 


(1933), Menzies (1937), Skinner 
(1938). While many other writers 
have shown the participation in con- 


Chained Conditioning 


Emphasis on kinesthetic stimuli 


as the basis of integration of the 
maze habit is probably due largely 
to Watson (1907). This emphasis 
was extended to other forms of 
learning by S. Smith and Guthrie 
(1921), who have continued to lay 
Stress on proprioceptive stimuli as 
integrators of movement, Guthrie 
(1935), S. Smith (1935), S. Smith 
and E. E. Fitch (1935), Carter 
(1936). 

Evidence showing that reaction- 
Produced stimuli may be used as 
conditioners is found in Hudgins 


ditioning of concomitant responses 
(Beck, 1939; J. Miller and Cole, 
1936; Schlosberg, 1932), chaining 
of conditioned responses has never 
been demonstrated unequivocally. 
Hull (1934c) interpreted the Miller 
and Konorski experiment as a type 
of chaining, since a tone which had 
accompanied passive lifting of the 
leg prior to feeding later caused both 
leg-lifting and salivation. The be- 
havior, however, lacks any definite 
serial character, whereby the stimuli 
resulting from one conditioned re- 
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sponse act as the instigators of a 
second act. Other chains of acts 
(Skinner, 1934) secure their serial 
character through environmental 
supports, and hence are not clear 
cases. Shipley’s experiment (1933, 
1935) is a type which requires more 
study. One interpretation of trans- 
fer in his experiment, as pointed out 
in Chapter 10 (p. 230), is the media- 
tion of a second response by the 
evocation of a first. This, if accept- 
able, represents genuine chaining. 


Maze Learning 


That the sensory control of the 
maze habit may be independent of 
kinesthesis has been shown by Lash- 
ley and Ball (1929) and by Honzik 
(1936). A review of the many fac- 
tors leading to maze errors and to 
their elimination may be found in 
Buel ( 1935). 

The more important maze studies 
directly or indirectly influenced by 
the attempt to apply or to criticize 
the deductive use of conditioning 
principles include: A. C. Anderson 
(1933), Ballachey and Buel (1934a, 
19346), Bruce (1937, 1938), Buel 
(1934, 1938, 1939), Buel and Balla- 
chey (1934, 1935), Dennis (1935), 

ennis and Sollenberger (1934), 
Drew (1939), Gilhousen (1938), 
Honzik and Tolman (1936), Hull 
(1934a, 1934b), Hull and Spence 
(1938), Hunter (1935b, 1936b), F. 
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N. Jones and Taylor (1938), Leeper 
(1935), Liddell (1934), Lumley 
(1932), N. E. Miller (1935), N. 
E. Miller and Miles (1935, 1936), 
N. E. Miller and Stevenson (1936), 
C. T. Morgan and Fields (1938), 
Reynolds (1936, 1939), F. L. Ruch 
(1934), Snygg (1936), Spence 
(1932), Spence and Shipley (1934), 
Spragg (1933, 1934, 1936), Tolman 
(1933), Zieve (1937). 


Serial Verbal Learning 


No attempt will be made to cite 
references to the voluminous lit- 
erature on serial verbal learning. 
Reviews and summaries may be 
found in Hunter (1934), McGeoch 
(1934). A classified bibliography is 
given in McGeoch (1933). 

Lepley’s (1932) hypothesis re- 
garding the applicability of condi- 
tioning to memorization was pro- 
posed originally in a seminar which 
Hull conducted. Among the studies 
relating conditioning specifically to 
memorization and retention may be 
listed E. J. Gibson (1938), Hov- 
land (1938a, 1938b, 1938c, 1939, 
1940), Hull (1935a, 1935b, 1937), 
Hull, Hovland, Ross, M. Hall, D. T. 
Perkins, and F. B. Fitch (1940), 
Lepley (1932, 1934, 1936), Patten 
(1938), Peak and Deese (1937), 
R. R. Sears (1936), Shipley (1939), 
Ward (1937), Witmer (1935). 


Chapter Io 
PROBLEM SOLVING 


HERE are many laboratory situations in which the interest 
ale of the experimenter is centered on the success achieved 
by a subject on the first trial in a novel situation, or on a 
successful performance which appears suddenly after repeated 
failure, and is thereafter sustained. Experiments in which 
behavior of this sort typically occurs are commonly called 
problem-solving experiments, and are designed to study the 
‘higher mental processes.’ 

Conditioning is primarily concerned with those instances of 
behavior modification which depend upon repeated trials. The 
effect of reinforcement can be tested only by a subsequent 
presentation of the same stimulus situation. Conditioning 
principles, therefore, appear not to be applicable to instances 
of insightful learning, since the term insight implies a kind of 
learning in which a conspicuous readjustment occurs in a 
single trial. The implied novelty or discontinuity with the 
previous history of the organism seems to contradict associa- 
tion or conditioning principles. Thus Guthrie has stated: “In 
so far then as insight means the ability to meet a new and 
unique situation with an adequate response it must remain in 
the category of luck” (1935, p. 195). A conditioning theory of 
problem solving obviously requires continuity of some sort 
with the past learning of the organism, but in order to use the 
concept of insight it is not necessary to assume that the dis- 
continuity with the past is complete. No believer in insight 
would deny that the insightful solution of a mathematical 


problem requires a knowledge of the symbols, conventions, 
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and operations characteristic of a particular branch of mathe- 
matics. Novelty enters as the events are structured in new 
relationships. The important thing to be stressed in insightful 
solution is that the possession of sufficient past experience to 
provide the materials essential to solution does not of itself 
guarantee the solution. One organism, given the same past 
experience and habit acquisitions as another, may not be able 
to solve a problem which the other can solve. Therefore a 
conditioning account of insightful learning must explain both 
how the raw materials relevant to the solution are learned, and 
how they are assembled in the way appropriate to the problem 
at hand. Such an account cannot be made easily in condition- 
ing terms, but there is no a priori reason to believe it impos- 


sible. 


Mediated Stimulus and Response Equivalence 


The clearest examples of problem solution are found in 
experiments which rule out the possible influence of repeated 
reinforcement of the correct response; that is, when solution 
is demanded on the first trial in a novel situation. A condi- 
tioning explanation must relate the correct solution in some 
way to the subject’s prior experience with the elements of the 
problem situation. Although the most hopeful suggestion for 
explanation through the concepts of classical conditioning is 
the phenomenon of generalization, simple sensory generaliza- 
tion alone is not adequate to account for the degree of novelty 
which exists in most problem situations. Many examples of 
insightful behavior can be redescribed as the appropriate 
combination of two or more previously learned experiences. 
The combination involves the mechanism of mediated stimulus 
equivalence (mediated generalization, cf. p. 201). 

An example of learned stimulus equivalence has been de- 
scribed by Brogden (1939c). A light and a bell were presented 
together to a dog for 200 trials, after which the dog was trained 
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to retract its foreleg to the bell followed by shock. On a subse- 
quent test it was found that the dog responded with leg re- 
traction to the first presentation of the light. This is not simple 
conditioning, because the light had never been followed by 
shock; it is not higher-order conditioning, because the light 
and bell were not combined after the bell had become a con- 
ditioned stimulus; it is not simple sensory generalization, for 
dogs without the previous light-bell trials did not respond to 
the light after training with the bell. An explanation in con- 
ditioning terms involves the inference that some reaction (not 
necessarily an observable or overt one) which is originally 
made to the bell becomes conditioned to the light in the light- 
bell trials. This hypothetical intermediate reaction may be 
considered the mechanism of the learned equivalence of bell 
and light in eliciting leg withdrawal. Brogden’s experiment 
gives no clue to the nature of the reaction. In the situation as 
described, the intermediate reaction serves no instrumental 
function in securing a reward or avoiding a punishment; it is 
comparable to what has been termed in other connections a 
perceptual response, an idea, a symbolic process, or a pure 
stimulus act. 

The possible rôle of intermediate reactions in learned stimu- 
lus equivalence is more clearly demonstrated in a recent ex- 
periment by Lumsdaine (1939) in which an overt reaction 
was studied. Following a procedure employed earlier by 
Shipley (1933, 1935), Lumsdaine established conditioned eye- 
lid responses in human subjects by presenting a light rein- 
forced by the blow of a mechanical striker on the cheek near 
the eye. A finger withdrawing movement was subsequently 
conditioned by combining the striker with a shock to the 
finger. After this course of training, it was found that the light 
evoked finger reactions in a large proportion of the subjects, 
although it had never been followed by shock. The records 
reproduced in Figure 33 illustrate the sequence of events in 





FIGURE 33 


Tue Lip REFLEX as AN INTERMEDIATE RESPONSE IN THE LEARNED 
EQUIVALENCE BETWEEN A LIGHT AND A STRIKER AS STIMULI FOR 
CONDITIONED FINGER RETRACTION 


The upper record shows a conditioned finger retraction (CR) after the 
striker has been repeatedly followed by shock. The eyelid reflex to the striker 
(Rs) is present incidentally, but stimuli produced by it may have become 
conditioned stimuli for finger movement. Conditioned eyelid responses had 
previously been conditioned by reinforcing a light with the striker. In the 
lower record is shown the result of presenting the light after the finger 
conditioning has intervened. The light elicits the conditioned lid response 

CR:) in accordance with the prior training, but perhaps because of stimuli 
related to the lid response, conditioned finger retraction (CRz) is also evoked. 
(Traced from unpublished photographic records, courtesy of A. A. Lumsdaine.) 
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which the lid response is considered the intermediate reaction. 
If it may be inferred that the proprioceptive stimuli from the 
reflex lid response to the striker became conditioned stimuli 
for finger withdrawal, any stimulus leading to lid closure might 
produce a tendency to finger withdrawal. Since the lid re- 
action had been conditioned to the light, the light may have 
evoked finger withdrawal by way of the stimuli from the lid 
movement. The situation is not, of course, quite as simple as 
this, and just any blink would not, after this training, cause 
finger movements. When other supporting stimuli are present 
(i.e., the familiar apparatus, the finger on the electrode, etc.) 
the proprioceptive stimuli from the lid may be sufficient to set 
off the response. Here the lid reflex to the striker was a defi- 
nite, vigorous response. In the experiment of Brogden men- 
tioned above, it may be assumed that something equivalent to 
this lid reflex occurred in response to the buzzer even though 
it was not observed experimentally. 

It can be seriously doubted that all learned stimulus equiva- 
lence is mediated by proprioceptive stimuli arising from inter- 
mediate reactions, as would be implied by a thoroughgoing 
motor theory of thinking and of consciousness. In Lumsdaine’s 
experiment (as in a related study by J. Miller and Cole, 1936), 
the finger response did not always wait for the eyelid re- 
sponse but in a few instances coincided with it or preceded it. 
This suggests the possibility that the eyelid response may not 
be an essential step in the sequence, but merely an overt indi- 
cator of a central ‘set’ which is common to both the light and 
the striker stimuli. Loucks (1931) has presented the most 
critical evidence on this point. He trained rats in a delayed 
temporal alternation problem which required a choice reaction 
on the basis of some sort of internal cue. Successful perform- 
ance was not destroyed when the rats were given a general 
anesthetic during the five minute period of delay before choice. 
Under such conditions it is not possible to assume that the 
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reaction was based on the proprioceptive stimuli arising from 
some intermediate response such as a maintained posture. 
More direct study of the mechanism of learned stimulus 
equivalence in animals has not yet yielded conclusive results 
(Loucks, 1939). 

Sign-learning Experiments. In an attempt to show the 
inadequacy of a conditioning theory of learning, Tolman 
(1933) performed an experiment which he designed expressly 
to test the relative applicability of his sign-learning (‘sign- 
Gestalt’) theory and conditioning. One group of rats were 
shocked in the food-box at the end of a locomotor problem. 
This produced retardation (i.e., avoidance learning) to be 
expected from conditioning theories. Another group of rats 
were shocked in the food-box, having been placed there with- 
out the usual run through the path. The question is whether 
the rats will respond appropriately at the beginning of the 
maze on the next trial. Conditioning theory, according to Tol- 
man, would predict no influence, since the maze running re- 
sponse was itself not punished. Sign theory would predict 
retardation, since the food-box would be signified as ‘bad.’ 
The rats behaved as Tolman thought they should according to 
conditioning theory; that is, their behavior was not affected 
by the separate punishment in the food-box. 

N. E. Miller (1935) objected both to Tolman’s analysis and 
to his experiment. He predicted that, according to condition- 
ing theory, rats should evidence anticipatory avoidance be- 
havior, even though they were shocked in the food-box with- 
out having run to it. He then set up an experiment in which 
he felt a better test could be made. At the end of a straight 
alley one group of rats found a reward device which consisted 
of a box which made the rats cramp their bodies to the right 
in order to secure food. Another group of rats found a device 
which required them to twist their bodies to the left in order 
to secure water. Food and water-deprivation were used as 
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appropriate conditions of motivation. Presumably the devices 
were sufficiently different so that fractional anticipatory re- 
actions to them would differ, whether these reactions depended 
upon the difference between anticipated food and water or 
upon the differences between right and left body positions. 
Half of the group trained to water and half of the group 
trained to food were now shocked outside the maze, but while 
utilizing the reward device. When they were next placed in the 
alleys, their running speeds were markedly reduced. When, 
however, the rats composing the other half of each group were 
shocked in the device not used in the maze, their running was 
much less affected. A third group, shocked under quite dif- 
ferent circumstances, showed no effects whatever upon their 
performance. 

These results are interpreted by Miller as demonstrating a 
learned equivalence between the reward device and the alley 
itself. The common element in the two situations is the frac- 
tional anticipatory goal reaction (r,) which occurs, after train- 
ing with food reinforcement, at the beginning of the alley as 
well as at the termination (p. 215, Chapter 9). The rg plays the 
rôle of the intermediate reaction, comparable to the eyelid 
response in Figure 33. When the animal is placed in the re- 
ward device for shocking, the r, is elicited as usual, and the 
internal stimuli arising from it become conditioned to the 
avoidance responses which were evoked by the shock. When 
the animal is subsequently placed at the start of the alley, 7» 
also occurs, producing the stimuli to which the avoidance re- 
sponses have just been conditioned. These responses interfere 
with forward locomotion, so that it is prevented or slowed. 
Comparison of Miller’s results with Tolman’s suggests that 
the distinctiveness of the goal reaction, and consequently of 
the fractional anticipatory goal reactions, is an important 
factor in determining the degree of equivalence of shock after 
running, and shock in the goal device not following running. 
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Delayed Reaction Experiments. In the direct method 
now commonly used in the study of delayed reaction, an ani- 
mal is shown two containers, such as inverted cups. Food is 
placed under one cup while the animal is watching. Now a 
screen is interposed, or the animal is taken away for a short 
interval of time. When the screen is lifted, or the animal re- 
turned, a choice between the two cups is permitted. Suc- 
cessful response consists in choosing correctly in a series of 
trials above chance expectancy. An analysis of delayed re- 
action performance along the lines of the experiments dis- 
cussed above would indicate that there is a learned equiva- 
lence between the cup on the test presentation, and the food 
stimulus. Past training has established an approach response 
to the sight of food and hence also to the anticipatory reaction 
(rz) aroused by the sight of food. When one cup is presented 
together with the food stimulus, the overt approach is pre- 
vented, but the 7, becomes conditioned to the cup stimulus. 
On the subsequent test when the food is hidden, the cup stimu- 
lus evokes the rọ which produces stimuli evoking approach 
movements. 

According to this interpretation of delayed reaction success- 
ful performance should be determined by three factors in 
addition to the inherent capacity of the subject: the strength 
of conditioning of 7, to the sign stimulus (cup), the strength of 
conditioning of approach movements to rg, and the degree of 
difference (i.e., the discriminability) of the two 7,’s evoking 
left or right approach movements. Although the interest in 
delayed reaction experiments has been directed predominantly 
toward comparison of the intelligence of various species (or 
of individual animals after cerebral damage) some data bear- 
ing on the factors listed above are available. The strength of 
conditioning of 7, to the sign stimulus is a function of the pre- 
liminary presentation. It should be noted that in the typical 
procedure the cup and the food are shown together for only 
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one trial. No systematic study has been made of variations in 
frequency of repetition of the preliminary presentation. On 
the other hand the strength of conditioning of approach move- 
ments to 7, is a function of training which took place before 
the experiment proper. Since the direct method utilizes a habit 
of approaching visible food, a habit that has been practised 
throughout the subject’s life, the conditioning has been found 
to be stronger than in the case of the indirect method in which 
training is given experimentally to establish an approach re- 
sponse to a stimulus such as a light. The third factor—the 
difference between the alternative r,’s—is of course not open 
to direct observation. They should differ in proportion to the 
difference in the final goal reaction of the left and right ap- 
proach responses. It has been found that performance is better 
on the basis of spatial cues than on non-spatial cues (Nissen, 
Riesen and Nowlis, 1938), and that it is improved by increas- 
ing the spatial separation of the two food containers. These 
points require much further analysis before a conditioning 
interpretation of delayed reaction becomes plausible. 

There is no explanation in the above analysis of the opera- 
tion of the limiting factor of delay interval. The commonly ob- 
served finding that delayed reaction becomes progressively 
less successful as the interval is lengthened from a few seconds 
to a few minutes probably depends upon an entirely different 
mechanism. Short delays are presumably bridged by the same 
process that is involved in short trace conditioned responses 
and short delayed rewards; namely, by whatever constitutes 
the stimulus trace. The maintenance of a posture, or the con- 
tinuation of the reverberating effects of some other muscular 
or neural activity, may be postulated as the mechanism of the 
shortest delays. Longer delays probably depend upon types 
of learned mediation which are not direct functions of the dur- 
ation of the interval. 


Reasoning Experiments. Maier (1929) has conducted a 
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series of experiments on reasoning in rats. After being taught 
the necessary segments of behavior for problem solution, the 
rats were placed in a situation in which the learned habits 
could not function directly, for the old segments of the task 
appeared in new relationships. If the animals could behave in 
accordance with the new relationships they would exhibit 
reasoning. This they appeared to be capable of doing. Hull 
(1935c), accepting the facts as demonstrated by Maier, has 
given a deduction conforming to his other analyses of learned 
behavior. The most important rôle is played by the fractional 
anticipatory responses. 

Hull proposes to account for the behavior that would occur 
in a conventionalized form of the Maier situation, as shown in 
Figure 34. There are first four sequences of acts separately 
learned, as follows: 

(1) Hungry; R to X, rewarded with food at X. 
(2) Hungry; U to X, rewarded with food at X. 
(3) Thirsty; R to U, rewarded with water at U. 
(4) Thirsty; R to H, rewarded with water at H. 

The problem which now tests the rat’s capacity for reason- 
ing is introduced. The rat, when hungry, is placed at R. Since 
it has been trained to run from R to X for food, it sets out in 
the direction of X. But a barrier has been interposed at B 
(Figure 34), so that this sequence is blocked. The choice of 
two paths now confronts the rat. Its tendencies to take them 
were previously equated, but then under circumstances in 
which it had been deprived of water. Now, not thirsty, but 
deprived of food, the question is whether the rat will choose 
the path R to U to X and reach food, rather than the path 
R to H. 

An anthropomorphic explanation would be as follows. The 
rat perceives and remembers the spatial relationships involved. 
It has learned two approaches to food, starting from R and 
starting from U. Therefore, it reasons, if there is some way 
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of getting to U, it can get to the food at X. But it has gone to 
U from R before, so it does so now. The reasoning in most 
reduced form is as follows: “The only alternative way of get- 


Ficure 34 
A ‘REASONING’ EXPERIMENT 


; Diagrammatic representation of a conventionalized form of Maiers 
reasoning’ experiment with rats, as used by Hull in attempting to deduce 
the behavior from conditioning principles. The runways are similar, but the 
platforms, R, U, X, and H are made as different as possible in form and in 


surface texture, so that they may be readily discriminated by the rat. 
(Hull, 1935c.) 


ting to X is via U; since I know how to get to U, I can get 
LOK” 

It is Hull’s problem to find a substitute for this ‘reasoned’ 
solution. His answer to the problem involves substituting in- 
ferred reactions for the understanding of relationships. The 
choice for the rat instead of lying between paths symbolized 
by ideas is between paths as represented by competing T€- 
action tendencies. There are two paths to choose between, SO 
that all that is necessary is to find an anticipatory reaction 
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which throws the balance in favor of the path R to U as 
against the path R to H. 

The broad outlines of Hull’s analysis may be indicated 
without recounting the details. Anticipatory tendencies are 
presumed to be aroused at the starting point R for each of the 
locomotor responses made on the several paths, as a result of 
the previous runs through each of the paths. It is assumed that 
these locomotor tendencies are equal, so that if based on them 
alone the choice of one path or another would be at chance 
frequency. Another set of anticipatory tendencies are con- 
ditioned to the drive stimulus. These are not alike for all paths, 
because in the earlier training the hunger drive functioned for 
paths R to X and U to X, the thirst drive for paths R to U and 
R to H. The anticipatory tendencies based on internal drives 
are conditioned appropriately to these different paths. The 
problem faced by Hull is to sum up the competing anticipatory 
tendencies when a hungry animal is at R, and the usual pre- 
ferred path R to X is blocked. The anticipatory locomotor 
tendencies are equal for the alternative paths R to U and R 
to H. Neither path has anticipatory goal reactions conditioned 
to the hunger drive, since training on both paths was with the 
thirst drive. Entrances into one or the other path would 
therefore be by chance, if some added tendency were not dis- 
covered favoring one over the other. This added tendency is 
one anticipatory reaction conditioned to the food drive that 
is common to path R to U, and not to path R to H. This is a 
tendency ry, based on the response to U when the animal, 
previously hungry, had learned to run from U to X. Since 
there is also an anticipatory tendency ry based on reaction to 
U when running from R to U while thirsty, these two rp s turn 
the balance, and the rat chooses path R to U instead of R to 
H. Once at U, the old habit of running from U to X when 
hungry is free to become active, and the problem is solved. 

When the difficulties of exposition are circumvented, it is 
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evident that the choice of pathway R-U-X depends entirely 
on the locomotion tendency evoked by the proprioceptive 
consequences of the anticipatory reaction ry. This tendency 
is greater than the competing reaction evoked by rg while the 
animal is at the starting point because the animal is hungry. 
It is the movement equivalent of the animal’s recollection that 
another way to get to food is by way of U, thus supplement- 
ing the tendency the animal already has to go to U. 

The question will immediately be raised whether the con- 
cept of reacting differentially to fractional anticipatory goal 
reactions is not as difficult to assimilate into a theory of be- 
havior as discriminating between perceptual representations. 
Can a starting point, such as U, on the way to food, be recog- 
nized as equivalent to a goal point, the same geographical U, 
when that place has been used as the terminal point for a 
thirst-quenching sequence? Hull’s answer ik, omei ~~ SEER, 
because the anticipatory reaction ry will be common to both 
situations.” At the present time this is a matter of conjecture, 
not of demonstration. It is merely attributing to the animal 
the capacity for equating in terms of its responses place U 
approached from one direction with one purpose and the same 
place when left in another direction for another purpose. The 
simultaneous presence of all of the anticipatory reactions is 
itself particularly difficult to account for on the basis of pro- 
prioceptive discrimination, although Hull (and Maier) at- 
tempted to make this easier by providing great differences in 
texture at R, U, X, and H. 

Since the proprioceptive consequences of fractional antici- 
Patory reactions have been made by Hull the basis for solu- 
tion both in this situation, and in others involving goal at- 
traction and purpose, it is important to examine the data bear- 
ing upon the organism’s Capacity to discriminate between its 


Own internal stimuli in situations comparable to those in which 
behavior is being deduced. 
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Hull recognized the importance of furnishing such evidence, 
and he demonstrated experimentally (Hull, 1933) that rats 
can be taught to turn one way when hungry, another way 
when thirsty, in an otherwise comparable external environ- 
ment. While the task was more difficult for the rats than 
naturalistic observations might have led one to expect, the 
demonstration was made satisfactorily. Leeper (1935) by 
modifying the experimental arrangements slightly, was able to 
secure the differentiation much more promptly. N. E. Miller’s 
(1935) experiment, reported above, is also pertinent. It is 
clear that anticipatory responses do occur, and that an ani- 
mal can differentiate between them. 

Habit-Family Hierarchy. An hypothesis to explain re- 
sponse equivalence in terms of conditioning has been presented 
by Hull (1934a) in his concept of the habit-family hierarchy. 
A family of habits is defined as a group of two or more re- 
sponse sequences, all of which may be initiated by a particular 
stimulus and terminated by a particular reaction. The indi- 
vidual habit sequences are still alternative in the sense that 
only a single member of the family can be active at once. At the 
Same time they are equivalent in that all are alike in bringing 
about substantially the same final goal reaction. A simple 
example of a habit-family is found in the many alternative 
spatial paths that an animal can take from one point on an 
unobstructed floor to another point at which food is placed. 
Hull has stated his hypothesis as follows: 


When an organism for any reason succeeds in solving a 
problem by a sequence of acts which is substantially that of 
one member of a habit-family, the other members of the fam- 
ily, particularly those occupying a more favored position in 
the hierarchy, will automatically become active in the new 
situation without any specific practice whatever. This transfer 
may be thought of as being mediated mainly through the an- 
ticipatory reaction (rg) which is common to all members of the 
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habit-family hierarchy and which therefore as a stimulus is 
conditioned to evoke the initial reactions of each member. 
(Hull, 1934a, p. 147-) 

According to this principle, after discovering food by a 
circuitous route in a free field, the animal would on the next 
trial tend to adopt a direct path. The habit-family hierarchy 
principle is thus a substitute for such field principles as least 
action. The tendency to take the direct path may account for 
multiple-path behavior of the rat in the maze (Dashiell, 1930), 
for goal-orientation in the maze (to the extent that it exists), 
and for the adoption of the shortest alternative pathway to 
the goal when the direct path is blocked. 

Circumventing a Barrier. Lewin (1933, 1935) has pro- 
posed a number of problems which arise particularly in the 
behavior of young children, when a goal-object is sought by 
the child, but a barrier makes a direct approach impossible. 
The behavior of the child is accounted for by Lewin in terms 
of valences and forces. For example, if the goal-object is at 
the center of a circular barrier, the child tends to circle around 
the barrier. This behavior permits locomotion without in- 
creasing the distance from the goal. If, however, the child is 
in the center and the desired object outside, the field proper- 
ties of the situation are such that running around the inside of 
the circle does not occur. Instead there is oscillation in a small 
arc at the side near the goal. The situation becomes more 
complicated when positive and negative valences appear to- 
gether, as when a desired toy floats on water which the child 
fears to approach. These situations are, of course, much freer 
in their possibilities than a maze in which the paths are fixed, 
and conditioning explanations appear to be proportionately 
the more difficult. 

Hull (1938) has analyzed the behavior which Lewin de- 
scribes by applying the principles of the goal-gradient and the 
habit-family hierarchy. He begins with a few basic assump- 
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tions such as that the goal-gradient hypothesis is applicable 
to visible goal-objects, and that the child possesses a habit- 
family hierarchy based on its earlier experiences. The most 
important experience assumed is that the larger the angle 
made by the beginning of the pathway with the straight line 
to the object, the longer the path to the object. From these 
assumptions many consequences flow. For example, too large 
a lure may increase the difficulty of a problem. This is known 
to be true from Kohler’s (1925) studies. Similarly, too strong 
a drive may make solution more difficult. This has also been 
demonstrated in certain problem situations (e.g., Yerkes, 
1934). Naive subjects, through the misleading action of the 
angular hierarchy, will under certain circumstances choose the 
longer of two paths. Certain of Lewin’s precise findings, such 
as the circling of the barrier, are also deduced. The goal- 
gradient hypothesis and the principle of the habit-family 
hierarchy, originally used in relation to maze behavior, have 
been found by Hull to be widely applicable to other behavioral 
situations, particularly those in which single-trial learning 
occurs through acquired response equivalence. 

Insightful Behavior in Typical Conditioning Experi-- 
ments. Instances of learned sensory and motor equivalence 
producing a novel response on the first presentation of a new 
situation have been recorded in connection with conditioning 
experiments. For example, it has been found true of human 
subjects who were trained to withdraw the right hand to a 
buzzer, that if the left hand was placed on the stimulating 
electrodes it was withdrawn the first time the buzzer was 
sounded (J. J. Gibson, Jack, and Raffel, 1932). This “solu- 
tion” evidently depended upon the equivalence of the original 
and the altered stimulus situations in evoking withdrawal 
response, and the equivalence of right hand and left hand re- 
- sponses. Similar examples of spontaneous transfer of con- 
ditioning effects have been reported in studies of the knee 
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jerk (J. J. Gibson and Hudson, 1935), and of leg retraction 
in dogs (Kellogg and E. L. Walker, 1938a). Wickens (1938, 
1939b) found that if the position of the hand in relation to 
the stimulating electrodes was reversed, the conditioned re- 
sponse changed spontaneously from extension to flexion. In 
these cases the equivalence is obviously learned, but data on 
the history of the component reactions is lacking. In the adult 
human subject the intermediate reaction is undoubtedly often 
a verbal one. 


Intermediate Reactions and Derived 
Reinforcement 


The theoretical possibilities discussed in the previous sec- 
tions show that the assumption of intermediate reactions may 
be used to account for several forms of problem-solving which 
appear, before analysis, to demand ideation or foresight. This 
suggests the desirability of reexamining the principle of ex- 
pectancy, which has been used to explain learning based on 
secondary reward and on the avoidance of punishment. It 
may be possible to reduce the explanation of derived rein- 
forcement to the principles of substitution and effect. 

When instrumental responses are reinforced by a secondary 
reward such as a token, there is no satisfaction of any primary 
physiological drive. The token, by reason of its repeated as- 
sociation with a primary reward, has acquired an effectiveness 
in reinforcing behavior in new learning situations. This ef- 
fectiveness has been termed derived reinforcement. It may 
safely be assumed that the token elicits fractional anticipatory 
goal reactions, but it is difficult to state how such rgs would 
have any reinforcing effect upon instrumental responses lead- 
ing to the token. White (1936) has suggested that reinforce- 
ment may best be described in terms of a ‘completion hy- 
pothesis.’ In any learning situation the organism possesses 4 


PROBLEM SOLVING 245 


certain need or want, which can be restated as the presence of 
anticipatory reactions relevant to the goal. The completion of 
the anticipatory reactions (by the goal response itself) tends 
to reinforce recent and concomitant stimulus-response con- 
nections. Thus, if the organism had learned to want the 
token, i.e., had anticipatory token reactions, the securing of 
the token would complete the sequence and be a reinforcing 
agent. By this conception, however, if the organism had come 
to expect shock in a certain situation, the occurrence of the 
shock would be reinforcing. Actually, of course, the opposite 
effect may result. At the present time, no satisfactory explana- 
tion of the operation of secondary reward in conditioning terms 
has been put forward. 

Can an explanation in terms of r,’s be applied to avoidance 
learning? An affirmative answer is intimated by the experi- 
ment of N. E. Miller (1935) previously reported, in which 
avoidance behavior was explained in terms of the anticipa- 
tion of reinforcement involving punishment. That the avoid- 
ance responses are aroused increasingly as the punishment is 
approached may be inferred from the gradient of avoidance 
reported by Bugelski and N. E. Miller (1938). This gradient 
can best be interpreted as the result of heightened anticipa- 
tory avoidance reactions as the noxious stimulus, e.g., shock, 
is neared. This heightening is what would be expected from 
the conditioning of 7,’s more strongly to stimuli close to the 
shock. If it be recalled that instrumental escape learning is 
a form of rewarded learning (‘safety’ being rewarding), a 
possible explanation of avoidance learning on the basis of 
heterogeneous reinforcement may be obtained. It need only 
be supposed that as the animal initiates a series of acts which 
have led to shock, the conditioned stimuli will arouse rgs which 
are anticipatory of the reaction to shock itself. These anticipa- 
tory reactions will conflict with the completion of the series of 
acts, and produce a state of heightened tension. This height- 
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ened tension is relieved by the avoidance behavior, as, for ex- 
ample, by running in the other direction. Since the animal 
succeeds in reducing tension, the situation has become dynami- 
cally that of escape learning, and the responses prior to the 
escape are strengthened by heterogeneous reinforcement. Just 
as rapid forward running is furthered by food at the end of the 
maze, so rapid running in the reverse direction is strengthened 
by the relief of the tensions aroused by the 7,’5. The strength- 
ened responses are those which conflict with the approach to 
the shock point. Hence in later trials, although the shock is 
not received, the avoidance behavior continues to be rein- 
forced, because escape inevitably reduces the conflict pro- 
duced by approach to the shock point. The derived reinforce- 
ment is therefore based on the escape from tension, rather than 
on the punishment. A similar explanation has been proposed 
by Mowrer (1939) who identifies the anticipatory rys with 
fear or anxiety. Any act which operates to reduce the anxiety 
will thereby be reinforced. In the case of both analyses, how- 
ever, there is no explanation of the commonly observed fact 
that the anticipatory responses (anxiety) seldom demonstrate 
any diminution through extinction, in spite of the fact that 
the painful stimulus constituting their reinforcement is not 
received by the organism. 

It must not be supposed that these conjectures represent 
solutions of the problems of derived reinforcement. They are 
offered as suggestions pointing toward the form which solu- 
tions in terms of conditioning may take. It has not yet been 
demonstrated that conditioning theories are able to give com- 
pletely satisfactory explanations of the processes involved. 


Problem Solving Capacity 


The attempts to reduce derived reinforcement to hetero- 
geneous or homogeneous reinforcement may or may not suc 
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ceed; certainly the contemporary accounts are not yet final. 
The fact that experiments clearly demonstrate the reality of 
learning based on secondary rewards and other kinds of de- 
rived reinforcement does not prevent controversy over in- 
terpretations. The token or sub-goal serves in some sense as 
a signal upon which the organism’s ‘expectations’ are based. 
There are those who believe that the ability to form expecta- 
tions based on signs may be deduced from the principles ap- 
propriate to homogeneous and heterogeneous reinforcement. 
There are others who believe that the capacity for expectation 
is one of the fundamental capacities of the organism, not to 
be explained away by reducing it to something else. The whole 
question is but one form of the familiar nature-nurture prob- 
lem, cropping out in a new context. Consider, for example, 
the controversy between Tolman (1937, 1938b) and Guthrie 
(1937). Associationism, which Guthrie endorses, assumes a 
minimum of native endomen Gestalt psychology, however, 
with which Tolman has some affiliations, is prepared to ac- 
cept a maximum of potentiality grounded in the nature of 
the organism, and maturing through the normal processes of 
growth (e.g., Koffka, 1924). In a reply to Guthrie, Tolman 
points out that Guthrie best explains all stupid behavior while 
he himself best explains all intelligent behavior. In a sense it 
is Guthrie’s task, as representative of association and condi- 
tioning, to explain how intelligent behavior is derived from 
stupid behavior, while it is Tolman’s task to show that be- 
havior is occasionally stupid only because circumstances have 
made the use of intelligence impossible. The argument is not 
over the relative amounts of stupidity and intelligence in 
actual performances (there is enough of each to keep the 
theorist occupied); the argument is over what is to be at- 
tributed to the natural endowment of the organism and what 
is to be attributed to experience. Even though an organism 
is stupid at birth, its acquisitions are not solely a matter of 
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experience, although it may know nothing but what it has 
learned. ‘There remains the problem why one organism profits 
by experience and another does not. Similarly, just because 
an organism can face a new problem intelligently, one need 
not deny that past experience plays a rôle in his solution of 
the problem. 

No matter how complete our knowledge of the history of 
specific abilities may become, at some point we shall always 
be forced back to certain primitive inherent capacities. At 
the lowest level these include such obvious capacities as de- 
pend on the possession of sense-organs; at the highest level 
they may include the possession of capacity for problem- 
solving. Meaningful discussions of the problems will demand 
the fullest possible history of specific abilities. 

What are the fundamental capacities which different writ- 
ers posit: Tolman (1932) finds it convenient to accept 
problem-solving capacities (e.g., means-end capacities) as 
among the givens. Both Guthrie and Hull, on the other side, 
admitting mature problem-solving abilities, do not accept 
problem-solving capacity as given. Both seek in anticipatory 
conditioned responses a means of deriving problem-solving 
behavior. According to Hull (1938) the fundamental capaci- 
ties which make possible the intelligent solution of problems 
are two: the capacity to react in proximity to a goal according 
to the goal gradient, and the capacity to form habit-family 
hierarchies. Both depend on the more fundamental capacities 
for the establishment of fractional anticipatory reactions, and 
for differential reaction on the basis of several concurrent 
anticipations. Thus one animal might be able to solve a prob- 
lem which another could not solve, although both had suffi- 
cient past experience available in the form of conditioned 
anticipatory responses. The more intelligent one might be 


able to discriminate better between its anticipations than the 
more stupid one. 
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Appraisal of Conditioning as an 
Explanation of Learning 


The present chapter and the previous one have been de- 
voted to discussion of attempts to relate conditioning prin- 
ciples to the problems of laboratory learning. The exposition 
has been intermingled with criticism, so that now a review 
may be desirable in order to furnish a basis for estimating 
conditioning as an explanation of learning. 

When we undertake to relate the specific facts found in 
conditioning experiments to the quantitatively stated conclu- 
sions from other learning experiments the difficulties turn out 
to be very great. It is necessary to assume, first of all, that 
instrumental acts may be conditioned by a reinforcing se- 
quence of behavior to which they lead; at the same time we 
must also assume more conventional conditioning in the form 
of redintegrative substitution. Then we encounter derived re- 
inforcement involving the expectancy principle, which adds a 
foreign and unfamiliar note. Without these supplementations, 
conditioning would remain an artificial experimental result, 
inapplicable to the large proportion of learning situations 
outside of the conditioning laboratory. With these modifica- 
tions, the way is opened for the systematization of a large 
proportion of learning experiments in terms of conditioning. 
Further assumptions with regard to the rôle of anticipatory 
responses, the gradient of reinforcement, the habit-family 
hierarchy, are needed in order to move from conditioning facts 
to the data of complex learning experiments. The most 
thorough attempts to foJlow the necessary steps have been 
made by Hull and his associates, much of whose work has 
been summarized in this and the preceding chapter. Their 
success has been sufficient to make it imperative upon all 
students of learning theory to give careful consideration to 
their proposals. Admittedly the limitations are great enough, 
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however, that greater service will be done by proposing con- 
structive alternatives than by confining arguments to specific 
proposals as they now stand in the literature. 

Two problems connected with the explanatory use of con- 
ditioning principles should be distinguished. The first prob- 
lem is whether it is possible to systematize theory so as to 
include all that is known about learning. The answer lies in 
the further elaboration of the kinds of deductions set forth 
in these chapters, until they include many more facts of 
learning. A good learning theory must not be contradicted 
by any of the known facts of learning, and it must have in it 
something relevant to each of the facts. It is obviously too 
early to decide how finally satisfactory such an extension of 
conditioning principles will become. 

The second problem is a more practical one. Can recom- 
mendations regarding training, with respect to the use of re- 
wards, punishments, intrinsic motives, the most economical 
methods of learning, and the like, be derived from systematic 
theory? Since the answers to practical questions of learning 
usually involve specific information as to capacities, motiva- 
tions and the prior habits of the subject, no matter how true 
abstract principles may be, they can be serviceable only 
within rather narrow limits. Thus a theory of maturation 
can never predict the actual vocabulary of third-grade school 
children in Kalamazoo, yet their actual vocabulary may be 
important if one is to make use of the maturational principle; 
e.g., in the selection of reading material. 

The emphasis on deduction in the extension of conditioning 
principles, while important for systematic theory, is to a cer- 
tain extent misleading. It must not be forgotten that predic- 
tion and control in psychology, as in other sciences, will be 
deductive only to a limited extent. Atomic numbers and tensor 
equations have not made superfluous the tables of empiri- 
cal constants found in chemists’ and physicists’ handbooks. 
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Therefore, while we seek relationships of the highest degree 
of generality, we must continue to study relationships of 
greater specificity. Laboratory data will never be superseded 
entirely by general principles. Even if systematization should 
be completed to the psychologist’s satisfaction, empirical re- 
lationships would continue to have an important status. 

It follows that the direct application of conditioning prin- 
ciples to practical problems involves the study of learning in 
specific practical situations. That is, experiments must be 
done with hunting-dogs and school children if pertinent ad- 
vice is to be given specifically for the training of dogs or the 
teaching of children. 

When everyday situations are examined carefully, it is 
soon discovered that they involve features which are very 
little represented in the literature of conditioning. Much prac- 
tical learning goes on in a social context, involving imitation, 
suggestion, emulation, rivalry. These cannot be studied con- 
veniently in the conditioned response situation, although the 
problem has been reflected in isolated experiments. It has 
been mentioned that it is necessary to keep a ‘social sheep’ in 
the room with the experimental animal. Here social factors 
are part of the undifferentiated background, and do not enter 
the data. W. T. James (1936, 1939) has reported that in ex- 
periments in which dogs were conditioned simultaneously, 
the social pattern of dominance and submission affected their 
conditioning as they stood in adjacent stands. The evidence 
drawn from learning outside of conditioning experiments, 
however, is so much richer than that within conditioning, that 
it would be foolhardy at present to attempt to regulate social 
behavior on the principles of conditioning. 

Many practical situations involve the organization of a 
number of acts into a smooth-running skill, as in the operation 
of a typewriter. Within the conditioning experiment there is 
no opportunity to study the detailed composition of such 
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skills, so that errors can be diagnosed and corrected. Such 
studies have to be made in the context of the skill itself. 

The learner often profits by guidance in the form of dia- 
grams and explanations in developing new modes of be- 
havior. How to improve such aids to learning, and how to 
profit from them in the most efficient manner cannot be 
learned from conditioning. 

Illustrations like these serve to show the futility of expect- 
ing conditioning principles (or other systematic principles of 
learning) to take the place in practical situations of analyses 
made within whatever concrete situation is under considera- 
tion. In order to give advice with respect to practical learning 
on the basis of conditioning we must be willing to use loose 
analogy, and to run the risk of extending conditioning prin- 
ciples to situations so remote from those in which they have 
been derived that their validity is questionable. The great 
detail required by Hull to give a trustworthy account of such 
relatively stereotyped behavior as maze running should give 
pause to anyone who attempts at the present time to offer ad- 
vice with regard to the formation and breaking of habits on 
the basis of known facts of conditioning. 


Notes 


Although insightful behavior is 
most conveniently studied in situa- 
tions requiring successful solution 
on the first trial, it is not limited to 
such situations. A subject may 
make repeated errors or random 


the possibility of insight. Krechev- 
sky’s emphasis on hypotheses in dis- 
crimination learning (p. 189) rep- 
resents an explanation in terms of 
insight to the extent that the hy- 
potheses arise from sources outside 


choices in the usual discrimination 
or trial and error situation, and then 
suddenly ‘discover’ the solution and 
react correctly thereafter. Because 
the same overt response has been 
previously made (by chance) and 
rewarded, it is difficult to separate 
the influence of reinforcement from 


the particular discrimination train- 
ing. Other writers have pointed out 
that the behavior of animals in the 
trial and error situation yields evi- 
dence of more intelligent behavior 
than is implied in the simple process 
of stamping in correct responses and 
stamping out wrong ones (€8+ 
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Adams, 1929). In a series of papers, 
Hull has derived many of the adap- 
tive characteristics of trial and 
error behavior in terms of condi- 
tioning principles (1930b, 1937, 
1939c). 


Mediated Stimulus and Re- 
sponse Equivalence 


The importance given to frac- 
tional anticipatory responses (rgs) 
in conditioning interpretations of 
problem-solving behavior cannot be 
over-emphasized. They are the sur- 
rogates for ideas in such condition- 
ing interpretations as those offered 
by Guthrie and Hull. In most of the 
deductions they are inferred, rather 
than observed. More experimental 
confirmations, in which the inferred 
intermediaries are observed and 
measured, are needed. 

A type of indirect evidence for 
anticipatory goal responses as in- 
tegrators of activity is furnished by 
those experiments in which a fore- 
taste of the reward facilitates learn- 
ing. According to a simple law of 
effect interpretation, such a fore- 
taste might interfere with learning 
through reducing the drive. On the 
other hand, by arousing rgs more 
vigorously, the behavior supported 
by 7,’s is furthered. That such pre- 
rewards aid learning is shown by 
Hull (1933), Bruce (1937, 1938), 
C. T. Morgan and Fields (1938). 


Field Theories and Condition- 
ing Theories 
Hulls proposal of the habit- 


family hierarchy as an alternative 
to certain aspects of field theory is 
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a definite attempt to meet the criti- 
cisms of conditioning theory as too 
analytical and too unresponsive to 
the present structure of the field 
(e.g., in relation to barriers and pos- 
sible alternative routes). The con- 
trast between Hull’s approach and 
that of field theory may be illus- 
trated by the alternative explana- 
tions of the rat’s choice of the 
shorter of two paths through a 
maze. According to Hull, this choice 
depends on prior conditioning, so 
that the entrance to the shorter 
path arouses more vigorous antici- 
patory responses than the entrance 
to the longer path, and the com- 
petition between the anticipatory 
responses determines the choice (cf. 
Figure 29, p. 217). According to 
the field principle of ‘least action,’ 
on the other hand, so long as the 
rat can discriminate between the 
longer and shorter paths, the choice 
is determined by the fundamental 
principle of animal nature which 
leads the rat to take the path re- 
quiring least energy expenditure in 
reaching its goal (Adams, 1931; 
Gengerelli, 1928; Helson, 1927; Tol- 
man, 1932; Wheeler, 1929). Un- 
fortunately, the principle of least 
action has not been based suffi- 
ciently on observations in which 
actual energy relationships are in- 
vestigated (Crutchfield, 1939). The 
same situation exists with respect to 
other so-called field principles. It is 
not entirely clear just at what 
points the field theories are in con- 
flict with conditioning theories of 
the type proposed by Hull, and at 
what points the two approaches, 
except for differences in exposition, 
are in essential agreement. It may 
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be noted, for example, that in apply- 
ing the goal-gradient hypothesis to 
the situation in which the subject 
perceives a goal-object at a distance 
beyond a barrier, Hull implies much 
of what Lewin means by valence 
and force in relation to distance 
(Hull, 1938; Lewin, 1933, 1935). In 
this connection, Hull’s own com- 
ments are pertinent (personal com- 
munication) : 

“As I see it, the moment one ex- 
presses in any very general man- 
ner the various potentialities of be- 
havior as dependent upon the si- 
multaneous status of one or more 
variables, he has the substance of 
what is currently called field theory. 
My habit-family hypothesis is pre- 
sumably a field principle in this 
sense. It is probable that my equa- 
tion expressing the goal gradient 
hypothesis (gradient of reinforce- 
ment) and even the one expressing 
the generalization gradient (gradi- 
ent of irradiation of Pavlov) might 
also qualify as bits of field theory. 
On the other hand, some might ob- 
ject to this usage on the ground 
that equations of this nature are not 
peculiar to gravitational or electro- 
magnetic fields but are almost uni- 
versal in natural science theory. 
Moreover the variables and the 
constants, together with their inter- 
relationships, which have so far 
appeared in equations expressing be- 
havior theories, seem to differ rad- 
ically from those which appear in 
the equations of physical field 
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theories. The notable lack of such 
equations in current Gestalt field 
formulations, while paradoxical, is 
probably symptomatic. For such 
reasons I prefer to be rather spar- 
ing in my use of the expression 
‘field theory’; if one has the sub- 
stance, the use of the term is scien- 
tifically unimportant however valu- 
able it may be for purposes of 
propaganda. But if one means by 
field theory what I have indicated, 
I am all for it and I see no inherent 
disagreement on this point between 
stimulus-response theory and Gestalt 
theory.” 


Conditioning and Learning 


Guthrie (1935, 1938) discusses 
learning rather informally in terms 
of conditioning and chooses many 
of his illustrations from everyday 
life. Hence there is some practicality 
in his suggestions for ordinary habit 
formation, but such suggestions are 
not necessarily grounded in the de- 
tails of laboratory conditioning. 

Hull’s program of explanation 1s 
much more formal, and now 10- 
creasingly mathematical. It repre- 
sents a particular type of scientific 
logic, and has been subjected to 
analysis in this respect, e.g., Adams 
(1937), Kattsoff (1939), J. G 
Miller (1939), Woodger (1938). 
The last two of these are attempts 
to formalize Hull’s system in ac 
cordance with modern symbolic 
logic. 


Chapter 11 
VOLUNTARY ACTION 


HE RELATIONSHIP between the conditioned response and 
‘che: voluntary activity has been the source of much argu- 
ment and confusion. The earliest studies of conditioning in 
human subjects raised the question whether the results in- 
dicated anything more than conscious, deliberate response on 
the part of the subject. Later studies have pointed out the 
importance of voluntary attitudes in determining the course 
of conditioning. On the other hand, Pavlov always considered 
the conditioned response essentially reflex in nature, and 
therefore to be understood without recourse to traditional 
concepts of volition and consciousness. In his sense condition- 
ing would be basic to the explanation of voluntary behavior, 
and some investigators have attempted to demonstrate this 
relationship in experimental detail. 


Degrees of Voluntary Control 


The term ‘voluntary’ has never been rigorously defined. 
If it is taken to mean activity under the control of the will, 
this leaves the mystery of the will to be explained. Neural 
definitions, which identify volition with control by ‘higher’ 
brain centers, are unsatisfactory because these centers are 
now known to be involved in types of automatic behavior 
which do not conform at all to the generally accepted mean- 
ing of voluntary. In common language, ‘voluntary’ refers to 
acts which are preceded by an idea or intention, acts of which 
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the individual foresees the consequences. They are acts over 
which the individual has the greatest degree of control; they 
are not compelled inevitably by a particular environmental 
situation. The knee jerk, for example, is classified as involun- 
tary because it depends so largely upon a blow to the patellar 
tendon. On the other hand, the extension of the arm to reach 
an object on the table is considered voluntary because the 
movement is quite independent of the particular external 
stimulus, and can be initiated or completely prevented by 
suitable instructions. Both movements may be the inevitable 
result of the total situation and the experience of the in- 
dividual, but the determinants cannot be described in the 
same way. 

Mere freedom from determination by specified external 
stimuli does not suffice to characterize volition, for certain 
types of self-initiated activity are not sufficiently under con- 
trol to be classified as voluntary. In a delirium, for example, 
responses may be quite unpredictable from the external situa- 
tion. They are self-initiated by the individual, but they are 
beyond that individual’s control. Certain types of self- 
initiated activity possessing little coherence or evident pur- 
posiveness, are spoken of as spontaneous movements, as in the 
random squirming and thrashing about of the newborn infant. 
The nature of the control which is called volition has eluded 
direct description. Introspective studies have discovered feel- 
ings of effort, kinesthetic sensations, determining tendencies, 
and so on, as correlates of voluntary activity, but such studies 
have not satisfactorily exposed the nature of the control. 

If voluntary behavior is defined in terms of factors which 
are subject to experimental control or measurement, it must 
be defined at present by exclusion; that is, as that form of 
activity which is least dependent upon specific measurable 
stimuli for its initiation and characteristics. In the laboratory 
voluntary behavior may be identified as actions which are 
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evoked, without repetitive training, by instructions given to 
the subject. Instructions to press a key when a light is flashed 
are, of course, stimuli, but they are not related to the re- 
sponse in the same way as the stimulus from which latency is 
measured. The instructions initiate preparatory actions which 
predispose to a certain response in the presence of an external 
stimulus which then serves as the signal (or trigger) to re- 
lease that response. 

Voluntary Action Compared with Reflex Action. If 
the characteristics of voluntary activity, as recorded in the 
laboratory, are measured carefully in comparison with reflex 
activity, it is found that there is no sharp line delimiting one 
from the other. Peak (1933a, 1933b) has demonstrated this 
clearly in a study of the relation between the voluntary wink 
and the reflex wink. She classified the possible differences be- 
tween them into the following three groups: (1) Descriptive 
features of the response. Although in general the voluntary 
wink has a longer latency, a greater amplitude, and a longer 
duration, there is considerable overlapping in the distribu- 
tions of the measures of the two responses. When the sam- 
pling of responses is increased, the distribution of any single 
characteristic tends to become continuous. (2) Determinants 
of the response. The voluntary wink is elicited following in- 
structions to the subject which are not necessary for the re- 
flex wink. On the other hand, the reflex wink is elicited by a 
loud sound which is not essential for the voluntary wink. But 
even this difference does not offer a clear-cut classification. 
The reflex will be modified in latency and amplitude if the 
subject is instructed to relax or to press a key when he hears 
the sound. And the voluntary wink will be influenced by the 
nature of the stimulus serving as a signal for the wink, decreas- 
ing in latency, for example, as the stimulus is increased in 
intensity. The stimulus and the instructions each influence 
both reflex and voluntary winks, and the distinction is one 
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of degree only. (3) Functional relations between the de- 
terminants and the response. Although the same determinants 
are effective for both reflex and voluntary responses, it is 
found that they influence the responses in a different way or 
to a different degree. For example, the duration of the open- 
ing phase of the wink is definitely modifiable by instructions 
in the case of voluntary but not of reflex winks. Again, the 
latency of the voluntary wink is markedly increased by in- 
structions to relax whereas the latency of the reflex wink is 
only slightly altered by the same instructions. 

Although the evidence indicates that no sharp distinction 
between the descriptive characteristics of reflex and voluntary 
action can be made, it is desirable to retain these concepts 
to designate the two extremes of a distribution of responses 
which differ in the nature and amount of their relationship 
to certain determinants. Reflex may be used to designate re- 
sponses whose determinants are most completely described 
in terms of experimental stimuli, while voluntary may be used 
to refer to responses which are relatively independent of the 
specific nature of the stimulus, but are easily modifiable un- 
der the influence of instructions or intentions. 

This criterion becomes very difficult to apply to the be- 
havior of animals and of infants without language. The dif- 
ferentiation of reflex and voluntary action is customarily made 
in such cases by analogy with similar behavior in human 
adults. 

Automatic or Habitual Acts. It is necessary to point out 
a further class of activities which are commonly called auto- 
matic or habitual. After hundreds of repetitions of the situa- 
tion in which an individual has unlocked his house door by 
turning the key to the right, the movement may become 
highly automatic and stereotyped. Automaticity is not, of 
course, a necessary consequence of extensive practice. When 
automaticity is achieved the stimulus situation seems to evoke 
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the response with complete regularity, for if the lock is 
changed so that the key must be turned to the left, the 
original response shows a strong tendency to persist. Such 
an automatic habit, then, is an involuntary action in the sense 
that it is determined by the stimulus situation in a regular, 
unvarying manner. But it was originally a voluntary re- 
sponse, so that it is distinguished from a reflex on the basis of 
its history. 

It is evident that the criterion of voluntary control is not 
a distinction between learned and unlearned. An act may be 
learned, and yet be involuntary. Thus William James called 
trained reaction-time activity “reflex pure and simple” (1890, 
I, p. 90). Involuntary automatic habits, based on activities 
once voluntary, may again be brought under voluntary con- 
trol. Hence a particular kind of performance may shift back 
and forth in the lifetime of the individual from relatively in- 
voluntary to relatively voluntary behavior, depending upon 
the extent to which it is under control. 


Degree of Voluntary Control of 
Conditioned Responses 


Where does the conditioned response fall on such a scale 
of involuntary-voluntary response? There is no categorical 
answer to this question. Conditioned responses may fall at any 
point along the involuntary-voluntary scale, depending upon 
the particular experimental situation in which they are es- 
tablished. The criterion of location on this scale must be 
the same as for any other variety of response: the extent to 
which instructions, sets, and attitudes affect the incidence, 
magnitude, and qualitative characteristics of the response. 

Experimental studies of the voluntary control of condi- 
tioned responses can therefore be carried out best with hu- 
man subjects who are responsive to instructions and who 
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can give reports on their attitudes. The question has been 
raised particularly with reference to the classical type of con- 
ditioning, involving substitution. In instrumental or avoid- 
ance training, with mature human subjects at least, the origi- 
nal response is itself voluntary, and the learning consists 
merely in discriminating the appropriate situation in which 
to perform the response. In this class would fall the bulb- 
pressing response used extensively by Ivanov-Smolensky with 
children, the reaction-time experiment, color naming, and so 
forth. 

The immediate problem at issue in the study of voluntary 
influences in conditioning is this: Do the principles of con- 
ditioning, derived from experiments on relatively involuntary 
responses in animals, suffice to account for the modifications 
resulting from analogous training in human subjects? If not, 
what further variables must be considered, and what are the 
laws of their operation? Incidental results from many experi- 
ments and a few more recent systematic investigations indi- 
cate that conditioned responses established by the classical 
training procedure in human subjects range from completely 
involuntary to completely voluntary. Some typical examples 
of the different degrees of voluntary control will be discussed. 

Involuntary Conditioned Responses. While involun- 
tary conditioned responses are likely to appear when the un- 
conditioned response is itself involuntary, it does not follow 
that the conditioned response will fall at the same point on 
the voluntary-involuntary scale as the unconditioned response 
on which it is based. It has been repeatedly demonstrated that 
the nature of the conditioned response is not completely de- 
termined by the unconditioned response. If the unconditioned 
stimulus has the characteristics of a reward or punishment, 
the subject may give an instrumental response which is al- 
ready in his repertory of voluntary behavior. If this compli- 
sation can be avoided by the use of a neutral unconditioned 
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stimulus, if the unconditioned response is quite independent 
of verbal control, and if the conditioned response occurs 
without the subject’s awareness, it appears justifiable to speak 
of the conditioned response as involuntary. One of the best 
examples fulfilling these conditions in adult human subjects is 
the conditioned pupillary response, since the unconditioned 
stimulus (illumination change) is not highly toned emotion- 
ally, and the subject is not aware of the response. Responses 
in very young infants, in primitive organisms, and in decorti- 
cate and spinal animals, are other examples of involuntary 
conditioned responses. Whatever influence voluntary factors 
may have in mature adults, at the other extreme there are 
conditioning phenomena independent of voluntary processes. 

Semi-Voluntary Conditioned Responses. Most of the 
experimental work on conditioning in human subjects has re- 
sulted in conditioned responses which fall into the broad 
category of semi-voluntary responses. Examples are condi- 
tioned salivation, knee jerk, galvanic skin response, and lid 
reflex, all of which are based on quite automatic uncon- 
ditioned reactions. The conditioned responses are not volun- 
tary because they cannot be perfectly duplicated in response 
to instructions, nor can they be completely prevented. Yet 
they are not involuntary because in all cases they are def- 
initely modifiable by the effects of instructions or of voluntary 
attitudes and sets. 

It has been found that the conditioned salivary response in 
human subjects is erratic and not predictable from the par- 
ticular properties of the stimuli and amount of training. Raz- 
ran (1935) found that although nearly all of his subjects 
showed some signs of conditioning, it was often sporadic and 
variable (cf. L. F. Jones, 1939). Thus one subject might 
give consistently large secretions, another almost none, and 
a third actually give a decrease in the flow of saliva in re- 
sponse to the conditioned stimulus. The same subject might 
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react differently when in the presence of a group than when 
alone, and results might be quite different on successive days. 
On the assumption that the nature of the conditioned response 
which a subject yielded was to be attributed to his attitude, 
Razran attempted to control the attitude experimentally by 
specific instructions. When subjects were directed to form as- 
sociations between the conditioned and unconditioned stimuli 
the responses were greater and more stable than when instruc- 
tions not to form certain associations were given. On the other 
hand, when the establishment of such determining attitudes 
was prevented by forcing the subjects to concentrate on the 
task of learning a manual maze while the conditioning stimuli 
were presented, the resulting conditioned response showed a 
regular progressive acquisition and stable reliable magnitude 
(Razran, 1936a). In later studies, by misleading the subjects 
with respect to the purpose of the experiment, Razran (19391) 
has been able to secure relatively involuntary salivary con- 
ditioning which much more nearly follows the rules to be ex- 
pected from Pavlov. 

The conditioned eyelid response, reinforced by the lid re- 
flex to sound or to cutaneous stimulation such as shock of 
air-puff, is a representative example of the conditioned re- 
sponse in human subjects. Under suitable conditions its de- 
velopment and extinction follow the regular and progressive 
course typical of the Pavlovian reference experiment. Volun- 
tary influences, however, are not without effect. For example, 
if the subjects are directed to make a voluntary hand move- 
ment whenever the conditioned stimulus is presented, the con- 
ditioned lid response is more rapidly and stably formed (J. 
Miller and Cole, 1936). Extinction resulting from the omis- 
sion of the unconditioned stimulus is slower when the manual 
response is still called for. As soon as the subject is told to 


Pease making the manual response, extinction becomes very 
rapid. 
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TABLE 7 


EFFECT OF SUPPLEMENTARY INSTRUCTIONS ON FREQUENCY OF 
ConDITIONED RESPONSES 


Conditioned stimulus, light; unconditioned stimulus, air-puff to one eye; 
interval 400 msec. The groups are arranged in order of increasing success 
of conditioning. Modified from J. Miller (1939b). 





Percent of trials 
on which condi- 
tioned responses 





Group Instructions appeared 
(1) Inhibitory “Be sure that you do not wink or 
start to wink before you have felt 
the puff” (n= 20) 26 
(2) Voluntary “Open your eyes each time the air- 
antagonism puff strikes your eye” (n= 20) 28 


(3) Noncommittal Only minimum instructions neces- 
sary for photographic recording 


(n =25) 38 
(4) Informed Told that light would be followed 

by air-puff (n = 20) 44 
(5) Voluntary “Wink voluntarily to the air-puff” 

supplementation (n= 20) Gy 

(6) Facilitatory “In case you feel your eyes closing 

or starting to close, do nothing to 

prevent it” (n= 20) 71 





* Subjects fell into two groups, as described in text. 


The effect of various types of instructions on the rate of 
acquisition and extinction of the conditioned lid response has 
been studied systematically by J. Miller (1939b). Some of 
his results under different instructions are summarized in 
Table 7. Efforts to inhibit response (Groups 1 and 2) reduce 
the frequency very little below the control groups uninstructed 
with respect to response (Groups 3 and 4). This suggests 
that the subjects in groups 3 and 4 had adopted an inhibitory 
set without specific instructions. Instructions not to prevent 
responses (Group 6) resulted in a greatly increased frequency 
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of response, and in slow extinction (Figure 35). The form of 
the responses showed clearly that they were not voluntary 
reactions. The results confirm the conjecture that conditioned 
eyelid responses normally develop against opposed inhibitory 
sets (Hilgard, 1938). The subjects instructed to wink volun- 
tarily to the air-puff (Group 5) apparently adopted one of 
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FIGURE 35 


EFFECT or Instructions on Course or ÅCQUISITION AND EXTINCTION OF 
ConpiTionep Eyer Responses REINFORCED BY A Purr oF AIR 


The three groups received different instructions, as reported in Table 7. 
The groups represented in the figures are (1) inhibitory, (3) non-committal, 
(6) facilitatory. The plotted values represent the mean magnitude of response, 
absence of response being included as zero value. After J. Miller (1939b)- 


two sets. Thirteen of the subjects attempting to wink to the 
puff, irrespective of the light, reacted promptly to the puf 
at a latency of simple reaction time (e7 msec) The other 
7 subjects evidently were concerned to discriminate between 
the light and the air-puff. The discrimination set was shown 
objectively by the longer reaction time of the voluntary winks 
to the puff (198 msec.). Conditioned responses were given 
on only 18 percent of the trials, while the 13 subjects who did 
not adopt a discrimination set gave conditioned responses On 
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75 percent of the trials. It is evident that the nature of the 
instructions can determine the degree of conditioning, even 
though the response is not completely under voluntary con- 
trol. Furthermore, uninstructed subjects cannot be assumed 
to be free of any determining attitudes. Apparently many of 
them were trying to prevent the appearance of any response 
to the light. 

The degree to which specific instructions can influence the 
conditioned eyelid response has been further studied in the 
discrimination experiment. The course of discrimination un- 
der the usual circumstances of a conditioning experiment was 
described in Chapter 8, Figure 24. In a study of the effect 
of instructions all subjects were conditioned on the first day 
to the positive stimulus, all trials being reinforced. On the 
following two days, during which the negative stimulus was 
introduced without reinforcement, the groups differed in the 
instructions given. The resulting characteristics of responses 
to the positive stimulus are summarized in Table 8. 


TABLE 8 


EFFECT or Instructions on Cuaracteristics oF Conpitionep Eyeuip RE- 
SPONSES TO Positive STIMULUS WITHIN A DISCRIMINATION EXPERIMENT 
(After Hilgard and Humphreys, 1938a, p. 301) 





Groups in order Characteristics of responses to 
of amount of voluntary positive stimulus 
supplementation of response Mean, 10 subjects in each group 


FREQUENCY AMPLITUDE LATENCY 
—PERCENT —MM. —MSEC. 





Instructed to refrain from responding to 

either ‘stimmunigee tt eee LO ee 55 13 448 
Instructed to refrain from responding to posi- 

tive stimulus but instructed to respond to 

the negative stimulus ................. 71 16 404 
Without instructions with respect to response 74 19 401 
Instructed to respond to positive stimulus, 

not to negative stimulus ............... go 35 319 
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Responses are found to show increasing frequency, increas- 
ing amplitude, and decreasing latency when they are arranged 
in order from voluntary restraint through no instructions to 
voluntary facilitation. This is evidence that instructions make 
possible some control over the process. On the other hand, 
voluntary restraint does not succeed in overcoming the posi- 
tive conditioning tendencies. It cannot be asserted that the 
responses are completely voluntary, any more than that they 
are completely automatic. The sharpest break is found, in 
fact, between the responses which are most completely vol- 
untary (i.e., those under instructions to respond) and the 
others which represent the more usual conditioned responses. 
Corresponding results were found for responses to the nega- 
tive stimulus. Except under direct instructions to respond, 
the negative stimulus always elicited responses at lower fre- 
quency and with smaller amplitude than the positive stimulus, 
but the same trends were found. Conditioned responses per- 
sisted in spite of voluntary effort to restrain them, and they 
increased when voluntary restraint was reduced as well as 
when there was added voluntary effort to respond. 

A report by Cole (1939) supplements the investigations re- 
ported above. He found that verbal instructions greatly in- 
fluenced the acquisition and extinction of conditioned eyelid 
responses, but he also discovered a limiting case in which 
verbal knowledge was ineffective. In a series in which rein- 
forcement and non-reinforcement alternated, the subjects 
could report the order being followed, but their eyelid re- 
sponses did not follow this order. 

A conditioned response based on an involuntary uncondi- 
tioned response, and itself relatively involuntary, may exhibit 
a certain amount of indirect susceptibility to verbal control. 
The conditioned galvanic skin response, for example, is a re- 
action of which the subject is quite unaware, and over which 
there is no direct voluntary control. Instructions to make oF 


VOLUNTARY ACTION 267 


to withhold the response are ineffective. Since the response is 
a correlate of expectancy of pain, however, indirect instruc- 
tions will cause its modification. If an untrained subject is 
told that he will be given a strong shock 2 seconds after a 
light is flashed, there will be a strong ‘conditioned’ galvanic 
skin response on the first presentation of the light (Cook and 
Harris, 1937; Mowrer, 1938). This is an indirect control; the 
subject already has a naturally acquired galvanic conditioned 
response to the expectation of shock, and this response may 
be ‘reinforced’ through instructions. If, instead of giving this 
information verbally, the light and shock are presented to- 
gether for a series of trials, the conditioned galvanic skin re- 
sponse becomes gradually stronger, and seems to be follow- 
ing the typical course of conditioning. It is possible that the 
mechanism is similar in both cases, since expectation of shock 
is built up gradually in the course of the repeated double 
stimulus presentations. 

Just because a response is influenced by knowledge of the 
expected stimulus we cannot infer that it is completely volun- 
tary. In the eyelid experiments it has been shown that subjects 
who were told that the negative stimulus would be given 
without reinforcement discriminated more promptly than 
those subjects for whom the negative stimulus was introduced 
without prior information regarding it (Hilgard, R. K. Camp- 
bell, and W. N. Sears, 1938). This more rapid discrimination, 
correlated with expectancv, did not result in ‘voluntary’ dis- 
criminatory responses, however, such as were made when the 
subject was told to react to the positive stimulus and to re- 
frain from responding to the negative stimulus (Hilgard and 
Humphreys, 1938a). There is a gradation in degree of con- 
trol, which permits a distinction to be made between semi- 
voluntary and voluntary responses. 

Voluntary Conditioned Responses. Several studies have 
been made of the responses occurring when the classical con- 
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ditioning procedure is carried out with a voluntary act as the 
unconditioned response. In this situation the ‘conditioned’ 
response is essentially the same as a false reaction in the 
usual reaction time experiment, the conditioned stimulus tak- 
ing the place of a ready signal. If we accept this analogy, cer- 
tain conditioning principles such as generalization, extinction, 
recovery, and disinhibition are found to be applicable (Hun- 
ter, 1938; Rexroad, 1936, 1937). Some of these modifications 
suggest that the responses have become relatively automatic 
with practice, and are to that extent semi-voluntary rather 
than voluntary. 

Direct comparison of conditioned responses based on vol- 
untary reinforcement has been carried out on the eyelid re- 
sponse (D. G. Marquis and Porter, 1939). If a stimulus (light) 
is repeatedly followed by a weak sound stimulus to which 
subjects have been instructed to wink, conditioned winking 
to the light may be established. Significant differences are 
found between this response and one established by pairing 
the light with an air-puff stimulus which elicits involuntary 
reflex winking. Conditioned voluntary responses are brisk, 
complete lid closures whenever they occur; conditioned in- 
voluntary responses are small and show recruitment. Curves of 
acquisition and extinction of conditioned voluntary responses 
do not show the gradual and progressive changes character- 
istic of conditioned involuntary responses. 

Under typical experimental conditions favorable for estab- 
lishing conditioned involuntary winking, it was almost im- 
possible to secure any conditioned voluntary responses. Analy- 
sis indicated that the adoption of a discrimination set by the 
subjects, evidenced by the long reaction time to the sound 
stimulus, was the effective factor preventing responses to the 
light. Reduction of the intensity of the light, elimination from 
the instructions of any mention of the light, and elimination 
from the experimental procedure of any isolated presentations 
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of the light abolished the discrimination set, shortened the re- 
action time, and greatly increased the incidence of conditioned 
voluntary responses. When the same changes in procedure 
were carried out with reflex reinforcement, however, the effect 
was just the opposite; i.e., the frequency of conditioned re- 
sponses decreased. These results indicate that different func- 
tional relationships hold for conditioned responses based on 
voluntary reinforcement as compared with reflex reinforce- 
ment. 

Similar characteristics have been found for conditioned 
finger responses established with a voluntary instructed re- 
action as the unconditioned response (Yacorzynski and Guth- 
rie, 1937; Wickens, 1939a). In general, such conditioned re- 
sponses tend to be identical with the voluntary response. 
There is no gradual progressive increase in amplitude during 
training; the response occurs in all-or-none fashion. The 
course of acquisition and extinction in terms of frequency is 
not progressive. The response is established in the first few 
trials and subsequently may increase or decrease or remain 
constant depending upon factors other than repetition. Ex- 
tinction usually occurs abruptly in one or two trials. Under in- 
structions, subjects can perform or prevent the response to 
the extent that any equally automatized voluntary response 
can be controlled. 

It is likely that the numerous investigations of conditioned 
finger withdrawal with shock reinforcement have actually 
been dealing with a conditioned voluntary response. This is 
particularly clear in those cases in which instructions to with- 
draw have been used (e.g., H. M. Wolfle, 1932), but it may 
be true even in those cases in which no instructions were 
given. The finger withdrawal to shock is not a reflex in the 
sense of the flexion reflex; it is just as voluntary and just as 
involuntary as rising from a chair after sitting on a tack. The 
latency of the response is of the order of voluntary responses 
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rather than of the extremely short flexion reflex, and some 
subjects have been found to misunderstand the demands of 
the experiment so completely as to leave their fingers on the 
electrodes throughout the shock. 

Conditioned responses established on the basis of with- 
drawal to shock show many of the typical characteristics of 
conditioned voluntary rather than conditioned involuntary 
responses. With human subjects it has usually been found im- 
possible to secure the gradual, progressive learning curves 
that have been supposed to be typical of conditioning. Instead, 
the conditioned response appears as an all-or-none reaction, 
with no progressive change in magnitude as the result of train- 
ing. It may occur on the first few trials and very often de- 
creases in frequency with further training. It is not identical, 
however, with the withdrawal response based on an instructed 
unconditioned response, as Wickens (1939a) has recently 
shown. Shock reinforcement results in more rapid and stable 
conditioning, and the conditioned response has a different 
form, a greater resistance to extinction, and less susceptibility 
to inhibitory instructions. It is apparent that no categorical 
distinction can be drawn between voluntary and involuntary 
responses. Under different experimental circumstances, condi- 
tioned responses differ in degree rather than in type. But a 
useful distinction may be drawn among the factors determin- 
ing the nature and strength of conditioning. The conditioning 
principles outlined in earlier chapters may be distinguished 
from a class of factors which are roughly grouped as volun- 
tary. 

The Effect of Voluntary Factors on Conditioning. 
With human subjects, the results of any conditioning experi- 
ment will tend to be complicated by the influence of voluntary 
factors. Since the laws of conditioning do not extend to such 
factors, the discovery and formulation of conditioning prin- 
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ciples depend upon the devising of experimental situations in 
which voluntary influences are impossible or are eliminated. 
Three types of procedure have been adopted in the attempt 
to eliminate as variables all factors except the stimulus-re- 
sponse relationships embraced in the laws of conditioning. 
In the first place, subjects can be selected which do not have 
the capacity for appropriate voluntary reactions. For this rea- 
son, animals and young infants have been found to be ‘good’ 
subjects. Second, a response may be selected for study over 
which the subject has a minimum of voluntary control. Pupil- 
lary responses, salivation, and other autonomic reactions fit 
this category best. Third, although the subject is capable of 
a degree of voluntary control over the conditioned response, 
the experimental conditions may be so arranged that this con- 
trol is not exercised. Most experiments with human subjects 
have attempted to realize this situation, although with varying 
success. For example, the subject may be given no knowledge 
concerning the purpose of the experiment, or of the stimulus 
sequences to be employed. Subjects may be selected who have 
not had previous experience with conditioning. The subject 
may be instructed to take a passive attitude, and not to try 
to control any responses he may make. Since any normal per- 
son is bound to take some attitude toward the rather unusual 
experimental conditions with which he is confronted, some 
experimenters have sought to distract the subject so that he 
would not notice the relevant features of the situation. The 
stimuli may be made to appear weak and incidental (D. G. 
Marquis and Porter, 1939); the subject may be given some 
material to read (Thorndike, 1932a); he may be given an- 
other irrelevant task to perform and misinformed about the 
true purpose of the experiment (Razran, 19391). The last 
method has been found particularly effective. Scott (1930) 
discovered that conditioning may be more effective in hypno- 
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tized than in normal subjects, possibly because of attentional 
changes involved. 

When voluntary factors do enter the conditioning situation, 
their effect is to establish response tendencies which interact 
and summate with the tendencies strengthened by the condi- 
tioning procedure. While the relationships are quite com- 
plex, as the data in Tables 7 and 8 reveal, the discontinuities 
are not great. The sharpest change is introduced when there 
is deliberate supplementation of response with the effort to 
react voluntarily to one or another of the stimuli. Information 
about the stimuli and efforts at restraint produce less striking 
changes. Voluntary restraint may produce only slight changes 
because such restraint already characterizes the usual condi- 
tioning situation. Thus J. Miller (1939b) found that an in- 
hibitory attitude assumed by uninstructed subjects in the 
eyelid conditioning experiment was apparent not only in the 
reduction of the frequency of conditioned responses, but also 
in a tendency to open the eyelids. This was shown by a cut- 
ting off of the duration of both conditioned and uncondi- 
tioned lid closings, and by a greater frequency of small lid 
opening movements. The correlation between frequency of lid 
openings and frequency of conditioned responses in 25 sub- 
jects was p =—.84. 

The interrelationship between voluntary factors and 
problem-solving behavior as discussed in the preceding chap- 
ter is apparent. The integration of acts by proprioceptive con- 
sequences appears in theories of voluntary control as well as 
in theories of problem solution. In human adults, the language 
mechanism adds so richly to self-initiated behavior that both 
problem-solving and voluntary action in man often appear 
qualitatively different from the behavior of sub-human ani- 


mals, but the distinctions may be quantitative rather than 
qualitative. 
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Acquisition of Voluntary Control 


Pavlov believed that a complete account of the conditioned 
responses of an individual would also be an account of his 
voluntary behavior. He made no specific attempt to show 
experimentally how voluntary control is developed, but he 
maintained that every action is inevitably determined by the 
complex of external and internal stimuli according to the indi- 
vidual’s previous experience. The conclusion is not self-evident, 
however, for as we have seen in the examples discussed earlier 
in this chapter, most conditioned responses, unless established 
by reinforcement with an already existing voluntary response, 
differ significantly from voluntary behavior. 

An experimental investigation of conditioning as an ex- 
planation of the acquisition of voluntary control has been re- 
ported by Hudgins (1933). Constriction of the pupil of the 
eye, a response over which an individual has no voluntary 
control, was reflexly elicited by strong illumination. The prob- 
lem was to condition this response to some self-excited stimu- 
lus such as subvocal verbalization, so that the response could 
be aroused whenever the individual wished. The first step 
was to condition the pupillary constriction to a bell stimulus 
which was presented simultaneously with a bright light. After 
125 to 225 trials a conditioned constriction was regularly 
evoked by the bell. Then the subject was instructed to squeeze 
a dynamometer with his hand when the experimenter gave 
the command “Contract.” This hand movement closed electri- 
cal circuits which presented the bell and light stimuli. After 
further training, it was found that the pupil constricted when 
the subject contracted his hand, even when the bell was omit- 
ted. The hand response was eliminated after about 200 more 
trials, leaving only the verbal command as a conditioned 
stimulus. Finally, the subject was required to repeat the ver- 
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bal command vocally, then by whispering, and finally sub- 
vocally. In several individuals it was thus possible to establish 
an ability to constrict the pupil to self-initiated sub-vocal 
verbalization of the word “Contract.” The same procedure 
was effective in training the subjects to dilate the pupil to 
the word “Relax.” The particular commands used had no 
significance aside from the training procedures, because a sub- 
ject could be trained to constrict the pupil to the word “Re- 
lax,” or to respond to nonsense words such as “sig” and “tok.” 

Although attempts to repeat Hudgins’ experiment have 
not met with complete success, the theoretical possibilities 
which it suggests are extremely valuable and should be in- 
vestigated further with responses of other types. Some of the 
phenomena have been confirmed with vasomotor responses 
(Menzies, 1937; Skinner, 1938, p. 114). Such experiments do 
not, however, offer a complete solution to the problem of vol- 
untary control. They demonstrate that an involuntary re- 
sponse can be conditioned to a stimulus already under volun- 
tary control, but they do not answer the question of what 
initiates the subvocal verbalization which evokes the pupillary 
or vasomotor response. 

In discussing the possibility of ‘voluntary’ control over au- 
tonomic activities, Skinner (1938, p. 112 ff.) points out four 
possible mechanisms by which such control may be learned. 
(1) The first consists in producing the autonomic response by 
exteroceptive stimulation. Thus, if a hungry man were given 
food whenever his blood pressure rose, he might resort to 
sticking a pin into his arm to produce that effect. This would 
not ordinarily be called voluntary control of blood pressure, 
but the blood pressure rises as a result of self-initiated stimuli. 
(2) The second case is similar to the preceding, with pro- 
prioceptive stimuli substituted for exteroceptive ones. The 
hungry man might run up and down the stairs a few times 
to increase his blood pressure in order to be fed. (3) The 
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third case consists in the production of the autonomic re- 
sponse by an exteroceptive stimulus which has been condi- 
tioned to it. Thus a state of excitement may be aroused by 
reading a book deliberately chosen to be exciting. The book 
contains material to which autonomic responses have been 
conditioned, and hence choosing the book is one form of self- 
stimulation to produce the emotional state. (4) The fourth 
case substitutes proprioceptive conditioned stimuli. For ex- 
ample, the subvocal recitation of poetry may produce the pro- 
prioceptive stimuli which have been conditioned to produce 
an emotional state which includes autonomic responses. Skin- 
ner points out that Hudgins’ experiment is concerned with 
the last two of these categories. 

No simple experimental demonstration of the acquisition of 
voluntary control seems possible, for the relevant events are 
buried deep in the complexities of the nervous system and by 
their very nature are not expressed completely in terms of 
external stimulus and overt response—the data with which 
the psychologist deals. A theory of voluntary behavior can do 
no more than indicate the manner in which the principles of 
learning, isolated in uncomplicated experiments, are also il- 
lustrated in the acquisition of voluntary control. 

An individual possesses a repertory of responses before 
there has been any opportunity for learning. These responses 
are not all of the same nature. Some, customarily called reflex, 
are evoked only by certain definite stimuli. Others, called 
spontaneous or operant, are capable of being evoked by 
a wide range and variety of influences including external and 
internal stimuli and some sort of central process which seems 
relatively independent of afferent stimuli. 

1 The acquisition of voluntary control implies two changes 
in the mechanism of responding. Spontaneous movements be- 
come more adaptive and purposeful, and both they and reflex 
Movements become elicitable by self-initiated processes as in 
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Hudgins’ experiment. Random responses, through trial-and- 
error learning, come to be evoked in appropriate situations. 
Since a response must occur before there is any opportunity 
for learned modification, and since every occurrence of a 
response is necessarily the occasion for some sort of learning, 
it follows naturally that the spontaneous responses, occurring 
under the greatest variety of conditions, will be most subject 
to learning. Arm movements, for example, will be conditioned 
to thousands of different stimulus situations, and the precise 
discriminations that will be formed are so complex as to defy 
any attempt at genetic description. Movements which have 
certain features in common, such as reaching movements, will 
be organized in relation to each other as ‘concepts,’ and rela- 
tively novel situations will evoke reaching on their first pres- 
entation because of this conceptualization. As the child ac- 
quires language, the word ‘reach’ is repeatedly associated 
with the arousal of these movements, and eventually he can 
respond to commands given by another person or to self- 
initiated instructions. Such behavior is voluntary by any 
definition that can be proposed. 

Experiments dealing with the refinement of spontaneous 
movements under instructions, and with the acquisition of 
voluntary control over originally reflex responses, contribute 
fundamentally to the understanding of mature behavior as we 
find it. Even the practical problems of developing ‘will-power 
may be treated more wisely as we advance toward this under- 
standing. At the same time, it must be recognized that far 
more progress must be made before it will cease to be ar- 
rogant of the experimenter to suppose that he has solved the 
age-old problem of volition. To discover that self-instructions 
may serve as conditioned stimuli for all manner of behavior 
(including that which is peculiarly reflex) does not yet ac- 
count for the origin and organization of self-instructions. It 
is the self-commands which can be made relatively inde- 
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pendent of present environment stimuli—often in the pres- 
ence of great obstacles, and in line with remote goals and 
purposes—which constitute the essential problem in volition. 
To date no conditioning experiment has dealt with the prob- 


lem as so defined. 


Notes 


Degrees of Voluntary Control 


Skinner points out the corre- 
spondence in some respects between 
operant behavior as he defines it 
and what has traditionally been 
called voluntary behavior (1938, 
Oh, WIA) 

It has been contended by some 
writers that conditioned responses 
should be classified as voluntary 
because their minimum latency cor- 
responds to that of voluntary re- 
sponses, even when the reinforce- 
ment is by an involuntary act. This 
does not follow, however, because 
latency may be a correlate of the 
neural complexity of the task, and 
neural complexity is not itself the 
criterion of voluntariness. The longer 
latency of conditioned responses as 
compared with reflexes and the cor- 
responding neural complexity do 
Suggest, however, the possibility of 
Sreater variability and greater con- 
trol, thus making most conditioned 
responses fall into the category of 
semi-voluntary activities. 

The designation of most condi- 
tioned responses as semi-voluntary 
arises out of their susceptibility 
to influence by verbal instructions 
or concomitant voluntary activity. 

ong the papers dealing with 


voluntary or attitudinal modifica- 
tion of conditioning may be men- 
tioned Cason (1934b), Cole (1939), 
Cook and R. E. Harris (1937), G. 
L. Freeman (1930), Grant (1939a), 
Hamel (1919), Hilgard (1938), Hil- 
gard, R. K. Campbell and W. N. 
Sears (1938), Hilgard and Hum- 
phreys (1938a), Hudgins (1933, 
1935), Hunter and Hudgins (1934), 
Menzies (1937), J. Miller (1939b), 
J. Miller and Cole (1936), Mont- 
pellier and Colle (1939), Porter 
(1938d), Razran (1934, 1935, 1936a, 
1939k), Schilder (1929), Schlosberg 
(1932), Steckle (1936), Steckle and 
Renshaw (1934), Wendt (1930). 

Some of the earlier studies of the 
conditioning of voluntary responses 
have been summarized briefly by 
Razran (1936b). Among the papers 
dealing with conditioning on the ba- 
sis of voluntary unconditioned activ- 
ity are: Cohen, Hilgard, and Wendt 
(1933), Cowan and Foulke (1934), 
Eaton (1937), Gantt (1938c), J. J. 
Gibson (1936), E. J. Gibson (1939), 
Hunter (1938), Kline and Kohler 
(1935), D. G. Marquis and Porter 
(1939), Peak and Deese (1937), 
Razran (1933b, pp. 71-73 for Rus- 
sian work on children). Rexroad 
(1936, 1937), Stephens (1934- 
1937), Wickens (1939a), Yacorzyn- 
ski and Guthrie (1937). 
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Acquisition of Voluntary Con- 
trol 


The experiments which were un- 
successful in the attempt to repeat 
Hudgins’ (1933) study on the de- 
velopment of the voluntary control 
of the pupil are those of Steckle 
and Renshaw (1934) and Steckle 
(1936). See also Hudgins’ reply 
(1935). The pupillary response has 
been successfully conditioned in ani- 
mals and man by Baker (1938), Ca- 
son (1922a), Kotliarevsky (1935), 
Ten Cate (1934b), Harlow and 
Stagner (1933), Metzner and Baker 
(1939) and there appears little 
reason to doubt that results similar 
to Hudgins’ may be reproduced. 

The mechanism by which condi- 
tioned responses to  self-initiated 
stimuli are established is not entirely 
clear. The various possibilities sug- 
gested by Skinner are important. He 
and Delabarre found, for example, 
that one of their subjects was pro- 
ducing vasomotor changes by in- 
voluntarily controlling his breath- 
ing. Here the semi-voluntary act of 
breathing was conditioned to verbal 
stimuli, and served as the inter- 
mediary for vaso-constriction. Cor- 
responding intermediaries may be 
found for the pupillary response, 
since the pupil is activated normally 
not only by changes in the amount 
of light falling on the retina but by 
emotional changes and by changes 
im accommodation and convergence. 
The latter changes are under some 
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measure of voluntary control be- 
fore the experiment begins. Vaso- 
motor responses are similarly 
aroused by respiratory, emotional 
and postural reactions which are 
capable of voluntary control. For 
other experiments dealing with the 
verbal conditioning of vasomotor 
responses, cf. Kotliarevsky (1936), 
Menzies (1937). 

The acquisition of voluntary con- 
trol over involuntary processes was 
experimentally studied by Bair 
(1901) before conditioned responses 
were discovered. He taught sub- 
jects to move their ears, a move- 
ment which was not under volun- 
tary control before the experiment 
began, and which was also not a 
‘spontaneous’ or ‘operant’ response. 
Two methods were used. One con- 
sisted in a course of training in- 
volving facial gyrations which in- 
cluded ear movements. Gradually 
excessive movements were reduced, 
and only ear movements remained. 
This is an interesting illustration of 
what is sometimes spoken of as the 
differentiating of a specific response 
out of earlier mass action. The 
second method consisted of stimu- 
lating the motor nerve while the 
subject was making the effort to 
move his ears, thus giving him the 
‘feel’ of the movement. This re- 
sulted in more prompt learning. In 
a sense, this might be thought of 
as conditioned, since a movement 
was occurring while the conditioned 
stimuli (self-aroused) were present. 


Chapter 12 
PERSONALITY 


NEw field of application for conditioning procedures and 
Price: was opened through some experimental ob- 
servations made by Shenger-Krestovnikova in Pavlov’s lab- 
oratory in 1914 (Pavlov, 1927, p. 291ff.). A dog was trained 
to salivate when a luminous circle was projected on a screen. 
After the conditioned response was well established, a dis- 
crimination was obtained between the circle and an ellipse 
with a ratio between the semi-axes of 2:1. The discrimination 
was acquired comparatively quickly. The shape of the ellipse 
was then approximated by stages to that of the circle, and the 
discrimination training continued. Finally an ellipse with a 
ratio of the semi-axes of 9:8 was reached. At this point the 
discrimination not only did not improve, but in three weeks 
of training it became worse. 


At the same time the whole behavior of the animal under- 
went an abrupt change. The hitherto quiet dog began to 
Squeal in its stand, kept wriggling about, tore off with its 
teeth the apparatus for mechanical stimulation of the skin, 
and bit through the tubes connecting the animal’s room with 
the observer, a behavior which never happened before. On 
being taken into the experimental room the dog now barked 
violently, which was also contrary to its usual custom; in 
short it presented all the symptoms of a condition of acute 
Neurosis. (Pavlov, Conditioned Reflexes, Oxford University 
Press, 1927, p. 291.) 

This discovery of experimental neurosis led not only into 
the field of psychopathology, but also drew attention to indi- 
vidual differences. Not all dogs reacted alike when discrimina- 
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tions became difficult. The problems of individual differences, 
of emotions, of psychopathology, and of mental hygiene, now 
have a substantial place in the literature of conditioning. 


Experimental Neuroses 


The functional disturbances which made his dogs unsuit- 
able for experimentation were labelled experimental neuroses 
by Pavlov by analogy with human neuroses. The neurotic 
person, compared with the usual normal individual, over- 
acts to some stimuli and remains apathetic in the presence of 
others, and to this extent the phenomena of human neuroses 
parallel those of the dog. On the other hand, superficial simi- 
larity in symptoms does not permit the underlying dynamics 
to be identified as similar, until more precise comparisons 
are made. It is preferable, for the present, to think of the 
experimental neurosis simply as a set of symptoms of dis- 
turbance appearing in conditioning experiments under cer- 
tain circumstances, and to postpone judgment about the re- 
lationship of such symptoms to those of human neuroses. 

Experimental Neurosis in the Dog. According to Pav- 
lov, experimental neuroses develop only in animals of ex- 
treme types, either the extremely inhibitable or the extremely 
excitable. There are three circumstances under which the 
neuroses are said to arise: the application of a strong condi- 
tioned stimulus, when the animal is accustomed to a weak 
one (overstrained excitatory processes); protracted inhibi- 
tion, as in excessive delays (overstrained inhibitory proc- 
esses); and clash of excitation and inhibition, as in a too diffi- 
cult discrimination. Results similar to those reported from 
Pavlov’s laboratory have been obtained in Gantt’s laboratory 
at Johns Hopkins (Gantt, 1936). The following observations 


made in his laboratory are described from an account by 
Loucks (personal communication). 
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Behavioral disturbances were studied in a pitch discrimina- 
tion experiment. When the pitches of the tones to be dis- 
criminated were brought too close together, symptoms of rest- 
lessness and disorganization developed. All the dogs showed 
occasional periods of excitement in which the signals would 
cause them to bark, back away from the food chute, and even 
reject food if the experimenter tried to feed them by hand. 
Of three dogs studied for several years, only one developed a 
lasting excitement, so that it was unsuitable for further ex- 
perimentation. This dog was seen by Dr. Andreyev, an asso- 
ciate of Pavlov’s, who regarded it as a clearcut instance of ex- 
perimental neurosis. It was ‘neurotic,’ however, only in the 
experimental situation. It behaved normally in its living quar- 
ters, and had a good appetite. The events leading up to ex- 
perimentation, such as entering the laboratory or fastening 
the salivary disks, brought on the state of excitement so that 
further training was impossible. The only evidence of ab- 
normality outside of the experimental situation was shown if 
the pure tones used in the training were sounded while the 
animal was eating outside the experimental room. The tones 
would cause it to drop the food, whimper, and even back up 
or run to a corner of the room. 

Experimental Neurosis in the Sheep. Liddell and his co- 
workers at Cornell have studied conditioned responses in 
sheep for a number of years. Neuroses have been observed 
Since 1926. Upsets were developed in the sheep by several 
methods. One was the overcrowding of metronome-shock 
combinations, with a 5 second interval of delay between the 
metronome and the administration of shock. Another was at- 
tempted differentiation of closely similar rates of the beating 
Metronome. Others included training with too long a delay, 
and the regular alternations of signals for shock and no- 
shock when separated by equal rest periods. 

A neurotic sheep may be ‘diagnosed’ by a number of com- 
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mon symptoms which distinguish it from a normal animal. 
Among these, Liddell (1938) lists the following: (1) The 
sheep resists being led from the barn to the laboratory. (2) 
When placed in the Pavlov frame with loops under its legs 
and with electrodes fastened to the left foreleg it cannot stand 
quietly, but exhibits a stereotyped form of hyperirritability in 
which persistent tic-like movements and sometimes tremor 
of the left foreleg are accompanied by sudden starts and ir- 
regular respiration. (3) Negative conditioned stimuli, actu- 
ally signals indicating no shock, elicit vigorous leg flexion. (4) 
The capacity to delay the conditioned response has vanished. 
(5) Administration of drugs produces abnormal effects. Sub- 
cutaneous injection of epinephrine decreases the vigor and 
number of leg movements which occur at the signal in antici- 
pation of the shock, while at the same time the movements of 
the reaction limb during the rest period are increased. Cortin 
has the reverse effect. (6) At the conditioned signal, the pulse 
rate is increased promptly and strikingly, persisting many 
minutes after the shock. This is quite unlike the reaction of 
the normal sheep. 

The neurotic behavior is evidenced also outside the labora- 
tory. The indicators used outside the laboratory were pedome- 
ter records of activity, 24-hour records of activity obtained 
by placing the animal on a large suspended platform, records 
of pulse, taken by a remote stethoscope, and records of respi- 
ration. Each of these indicated an upset, excited state. Pulse 
and respiration were much more irregular than normal; the 
diurnal variations in activity and rest were very atypical. Nor- 
mal animals have long periods of rest during the night; the 
neurotic animals showed the same restlessness at night as dur- 
ing the day. Vacations from the laboratory temporarily re- 
stored the normal rhythm of rest and of activity, but the 


deviations returned as soon as the animals were again working 
in the laboratory. 


PERSONALITY 283 


Of five cases which have been observed (there is no state- 
ment of the proportion of sheep which these represent) all 
have continued in the neurotic state for five years or more. 
One animal, which developed a neurosis at the age of 4.5 
years, died nine years later, at 13.5 years, having remained 
in its disturbed state to the end. Two others were observed 
for six years in the disturbed state. While the animals might 
appear normal after a long rest, resumption of the condition- 
ing tests always precipitated the characteristic abnormalities 
of overreaction within a few days. 

The neurosis in these animals showed a remarkably uni- 
form course of development. A premonitory non-reactive or 
inhibitory phase of several days’ or weeks’ duration was fol- 
lowed by the abnormalities of activity, respiration, and pulse, 
as described. 

Experimental Neurosis in the Pig. The following ac- 
count of a pig trained by G. F. Sutherland, is reported by Lid- 
dell (1938). Shortly after weaning the pig was taught to run 
on a leash. Resentment was overcome gradually until the 
animal could be led around the pasture and finally into the 
experimental enclosure. At each stage of environmental re- 
striction the pig was likely to exhibit tantrum behavior be- 
fore it became tractable. It was taught to nose up the lid of a 
box to secure pieces of apple, and finally to stand quietly in 
a harness before this box for experimental purposes. A spring 
electrode was attached to its foreleg. Strong shock caused it 
to flex its foreleg, but 200 combinations with a tone did not 
result in conditioned flexion. Instead of struggling or squeal- 
ing when shocked, the pig would lift the lid of the food-box. 
In other words, whatever happened in the experimental room 
was taken as a sign of food. 

The situation was gradually made more complicated. The 
pig was fed and shocked on alternate days, feeding and shock- 
ing being preceded by discriminable tones. The pig was 
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noticeably upset on shock days, but quiet on feeding days. 
Between tests the pig spent much time opening the lid of 
the box. This is interpreted as a tension-reducing mechanism, 
similar to the sheep’s nibbling of the dry leaves in a maze 
when there is no other substitute activity. So far the animal 
could not be considered neurotic. 

In order to reduce the frequency of the approaches to the 
food-box between signals, the pig was now shocked if it 
raised the lid before the apple was delivered. Finally the de- 
livery of the apple was withheld until the pig raised the lid. 
It was therefore necessary for it to ‘make a decision’ to open 
the box, to risk shock on its nose if it was wrong, to receive 
the apple if the choice was correct. The conflict was now too 
strong, and the pig developed a neurosis. It delayed response 
for a long time, occasionally as long as an hour. During this 
time its eyes would be closed, its snout resting on the screen 
of the enclosure. For many minutes it would maintain a rigid 
posture with occasional trembling and growling. During the 
period of immobility a piece of apple could be balanced upon 
the pig’s snout without the animal attempting to remove it. 
Opening of the food-box, when it occurred, was impulsive. 
At the time of Liddell’s report the pig refused on the food 
day to open the box at all or to eat the pieces of apple as they 
fell, even when the lid of the box was propped open. 

Outside the laboratory the pig’s behavior was also changed. 
It had formerly been friendly and docile at all times. Now it 
kept away from the other pigs, frequently fighting with them 
and occasionally attacking the attendant when he entered the 
pig pen. 

Experimental Neurosis in the Rat. Until quite recently, 
attempts to produce experimental neuroses in the rat have 
been rather unsuccessful. Rats would keep working under 
quite adverse circumstances, and although occasionally re- 
fusing to run in the maze, or taking excessive amounts ot 
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‘time out’ in order to wash an already thoroughly washed 
face, they were seldom rendered unusable as experimental 


animals. 

Successful development of experimental neurosis has been 
reported by Cook (1939b). From among a group of six rats, 
three developed disorders of behavior in a situation described 
as follows: 


The rats were strapped to a stand so that the only sizeable 
limb movement possible was a flexion of the right foreleg. Un- 
der certain conditions such a flexion was rewarded with a food 
pellet; under other conditions it was punished with an elec- 
tric shock. Observations indicated that the animals experi- 
enced two principal stresses: the first, when they were re- 
quired to delay the food-bringing flexion until they received 
a bright-light stimulus; the second, when they were required 
to make a very difficult discrimination between a bright-light 
stimulus which permitted a food-bringing flexion and a dim- 
light stimulus which prohibited such a flexion on pain of 
shock. Because, under such conditions, the organism was re- 
celving simultaneously stimuli to the excitation and inhibi- 
tion of the same response, the stresses experienced have been 
characterized as comprising a “clash” between the neural 
activity of initiating a response and the neural activity of in- 
hibiting the same response. (Cook, 1939b, pp. 307-308.) 


Cook points out that restraint of activity is an essential feature 
of the experimental situation leading to abnormal behavior. In 
three other conflict problems which permitted more freedom in 
bodily activity no disturbance was produced. 

A distinction between nervous, agitated behavior, often 
observed in difficult discrimination or conflict situations, and 
a definitely abnormal breakdown has been drawn by Maier 
(1939b), who would restrict the use of the term ‘experimental 
neurosis’ to the latter. Maier trained rats in a discrimination 
Problem with the Lashley jumping apparatus. When the rats 
developed resistance to jumping toward the stimuli, they 
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were forced to react by shocking them or by directing a stream 
of compressed air against them. In certain cases a striking 
pattern of abnormal behavior was induced. During the initial 
active phase the animal ran violently and exhibited convul- 
sions and tics. In the succeeding passive phase the animal was 
apparently in a coma and could be molded into almost any 
posture. It is very significant to note, as Maier points out, that 
this ‘seizure’ occurred only when the compressed air was em- 
ployed to force jumping. In control observations it was found 
that the sound of the escaping air itself would produce the 
seizure in some cases in the absence of any conflict situation. 
Similar seizures have been induced in mice by the sound of 
jangling keys (Dice, 1935), and in rats by the sound of es- 
caping air (C. T. Morgan and J. D. Morgan, 1939) or of an 
electric bell (Humphrey and Marcuse, 1939). The rôle of 
the conflict situation in the induction of the seizure is not 
entirely clear at present. Maier found that seizures were more 
frequent when the rats were facing the discrimination prob- 
lem, but the number of animals was too small to permit a 
definite conclusion. 

Experimental Neurosis in Man. Social considerations 
make the deliberate development of experimental neuroses in 
human subjects out of the question, but it is possible to pro- 
duce minor upsets in laboratory situations which are some- 
what comparable to the animal experiments. ‘The best sum- 
mary in English of cases studied by Pavlov’s methods is that 
of Krasnogorski (1925), who described a number of experi- 
ments by other workers in his laboratory. 

A child of 6 years had a motor conditioned response to 4 
metronome beating 144 times per minute. This was success- 
fully discriminated from 92 beats and from 108 beats. Minor 
symptoms developed as the negative stimulus was increased 
in rate to approach the positive one. When the discrimination 
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had to be made between 144 and 120 beats per minute the 
child was described as “taciturn,” “refuses to go to the 
laboratory,” “walks and mounts the apparatus slowly.” The 
discrimination was now made more difficult by increasing 
the rate of the negative stimulus to 132 beats per minute. 
Positive and negative stimuli were alternated. The discrimina- 
tion broke down completely, and responses were given to both 
stimuli. The child was now described as “rude, fights, dis- 
obedient, excited, yawns, closes eyes, falls asleep.” 

Other reports are concerned with the behavior of pathologi- 
cal children. An experimental neurosis was developed in a 
cretin by attempting too fine a differentiation between neigh- 
boring points on the skin. A rowdy 6-year-old post-encephalitic 
boy was conditioned to a metronome which was reinforced by 
food. A tactile stimulus combined with the metronome pro- 
vided a negative combination which was not followed by 
food. Ordinarily the repetition of negative stimuli leads to 
drowsiness or experimental sleep, but in this case the child 
became very excited, cried, and breathed irregularly. 

Luria’s (1932) work shows that the conflicts which are 
aroused in free association experiments may also be revealed 
in disorganization of movements. While his work is not con- 
cerned with conditioning, strictly speaking, it suggests that 
personality disturbances may well be reflected in the kinds 
of measurements studied within conditioning experiments. 


Theories of Neurosis 


Theories of neurosis have been related to conditioning in 
two ways. Certain theories arose directly from the observa- 
tions on experimental neurosis. Pavlov’s and Liddell’s theories 
are of this variety. Other theories of clinical origin have been 
correlated later with the facts of conditioning. Chief among 
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these are a number of attempts to relate the theories of psy- 
choanalysis to the work of Pavlov. These two varieties of 
theories merit separate discussion. 

Theories of Neurosis Arising out of the Experimental 
Work on Animals. Pavlov’s theory of neurosis rests on his 
conception of the fundamental antithesis between excitation 
and inhibition in the cortex. The processes of inhibition and 
excitation are said to interact both spatially and temporally 
to result in a variety of states ranging from extreme excitation 
to extreme inhibition somewhat as follows: extreme irritabil- 
ity, when an inhibitory process becomes very difficult or 
impossible; normal waking state, representing an equilibrium 
between the processes of excitation and of inhibition; a state 
of equalization, when all stimuli, independent of their intensi- 
ties, act exactly equally; the paradoxical phase when only 
weak stimuli act and when strong stimuli have either no ac- 
tion at all or a barely noticeable effect; the ultra-paradoxical 
phase, during which only the previously elaborated inhibitory 
agents have a positive effect; finally, complete inhibition 
which Pavlov identifies with sleep. It is the isolation and fixa- 
tion of the transitory intermediate states which Pavlov be- 
lieves to mark the pathological condition. He summarizes all 
pathological instances as follows: 


- it is a difficult collision, an unusual confronting of 
the two opposing processes of excitation and inhibition (be it 
in time or intensity relations or even in both together), which 
leads to a more or less permanent destruction of the normal 
balance existing between these two processes. (Pavlov, 1928, 
p. 361.) 

Whether or not one is willing to follow Pavlov’s doctrine 
of excitation and inhibition (see Chapter 13), it is evident that 
the experimental neuroses which he observed depended upon 
a state involving conflicting tendencies. Liddell has attempted 
to describe this conflict without reference to its neurological 
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locus. He believes that the training to stand quietly in the 
harness is the behavioral side of the learned inhibition. When 
the discharge of excitation through the channel of positive 
conditioned responses becomes too difficult, the dammed up 
excitement interferes with the practiced quiescence, and neu- 
rotic behavior results. In a situation permitting free loco- 
motion (such as the maze) neurotic behavior does not arise 
because alternative methods of discharging the pent-up excite- 
ment are available. Pavlov noted that the recovery of animals 
was aided by freeing them from the experimental stand. The 
following illustration from the Cornell laboratory supports 
this interpretation. An animal which had developed a neurosis 
under the restraint of the conditioning situation was subse- 
quently tested in a situation free from restraint (personal com- 
munication from Dr. Richard Parmenter). The sheep was 
placed in a pen five feet square on the floor of the experimental 
room, where it was free to move about. In this situation the 
objective signs of the neurosis disappeared; for example, the 
tic movements of the foreleg were not present. “The animal 
appeared curious rather than alarmed at the sound of the 
buzzer, and his actions were those of avoidance and not 
panic. He received the shock calmly, lifting his leg rather 
slowly and the hard kick previously noted was absent.” This 
change, however, was in no sense a “cure,” for on the follow- 
ng day the animal, when tested in the original restraining 
situation, manifested the neurosis in all its former severity. 
The facts of experimental neurosis are unquestionably of 
great importance, and more detailed experimentation is ur- 
gently needed. On the other hand, the symptoms shown by 
animals and human patients are not identical, and the con- 
flict situations which are revealed in the psychiatric interview 
are not readily reduced to Pavlov’s formula. In his later years, 
Pavlov identified the results on animals with some of those 
clinically observed in man. Clinical observations of the effects 
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of frustration and conflict in human subjects show behavior 
somewhat similar to that observed in the experiments, notably 
hypertension, disorganization of response, and, occasionally, 
somnolence. The motivational organization within condition- 
ing experiments, however, is not sufficiently complex to include 
the more important conflicts which involve personal-social 
motives in a complicated culture. To include these conflicts 
and the symptoms which are related to them requires processes 
of analogy and deduction not yet justified by conditioning 
facts. 

At the present time the significance of the study of experi- 
mental neuroses lies chiefly in the procedure which has been 
developed for creating behavior disturbances experimentally in 
animals. In general, the method involves the conflict of oppos- 
ing action tendencies. The tendencies need not necessarily be 
established by laboratory conditioning procedures; they may 
have been acquired in the course of natural learning or they 
may be highly practiced voluntary reactions. The particular 
advantage of conditioning for the study of conflict is that the 
strength of the tendencies can be controlled and measured in 
terms of the experimental reinforcement. The laboratory study 
of conflict and resulting disturbances has shown the possi- 
bility of such work; further investigation must deal with the 
nature of the disturbance, the conditions under which it ap- 
pears, and the possibilities of therapy and reeducation. 

Attempts to Correlate Conditioning with Psycho- 
analytic Doctrines. Among a number of attempts to har- 
monize Freud and Pavlov, a paper by French (1933) may 
serve to illustrate the more serious efforts to estimate the pos- 
sibilities and limitations of a coordination between psycho- 
analysis and conditioning. The problem of positive associa- 
tion causes French no trouble, for the analogy between the 
establishment of an affective association and the development 
of a conditioned response is readily acceptable. Symptom 
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formation may be based upon generalization and induction. 
The studies of conditioned emotion make the parallel a 
plausible one between conditioning and the relatively unin- 
telligible associations which patients form under stress. The 
more difficult problem, and the one crucial to an acceptance 
of a parallelism between Freud and Pavlov, is to relate condi- 
tioned inhibition to Freudian repression. There is a possible 
parallel between the two types of inhibition (external and 
internal) and two kinds of repression, that caused by an ex- 
traneous motive such as fear (corresponding to external in- 
hibition), and that caused by a failure of gratification (cor- 
responding to internal inhibition). The difficult problem is to 
account for ego-manifestations, as in wishes, sublimation, and 
so on. According to French, the problem can best be en- 
visaged in terms of discrimination. Sublimation involves a 
discrimination between that which is socially acceptable and 
that which is not. This discrimination may become more diffi- 
cult if there is an increase in drive. It is known experimentally 
that increases in drive may make problem-solution more dif- 
ficult (Chapter 10). In psychoanalytical experience this is il- 
lustrated, for example, by the difficulty encountered in nor- 
mal heterosexual adjustments at puberty. Adjustment to 
reality may be thought of as a matter of conditioning and 
discrimination. French is aware that most of his analogies 
with conditioning involve situations more similar to Thorn- 
dike’s reward and punishment learning than to Pavlov’s rein- 
forcement. They are illustrations of Freud’s pleasure principle 
rather than of his repetition principle, and it is the latter 
which corresponds more closely to Pavlov. 

R. R. Sears (1936) has subsequently presented an account 
of repression in conditioning terms which differs from that 
of French because he does not assume a parallel between any 
single Freudian principle and a Pavlovian principle, but at- 
tempts to analyze the mechanism of repression into its com- 
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ponent stimulus-response sequences. The phenomena of re- 
pression may be illustrated by the order of events in Morton 
Prince’s well-known bell-tower case (Prince, 1921, p. 389ff.). 
A young woman was praying in a hospital room for the suc- 
cessful outcome of an operation being performed on her 
mother in the room below. Outside the window she looked 
upon a church-spire in which a bell was tolling. In her dis- 
tressed condition she was greatly annoyed by the bells, par- 
ticularly because they symbolized death. Her mother died. 
She later developed a phobia of bell-towers which made her 
avoid the streets on which churches were located. She had 
forgotten the source of the fear, but the story was later re- 
covered through automatic writing. The primary repression 
arose because of the great significance of the mother’s death 
in the daughter’s emotional life. It had become intolerable for 
the daughter to dwell upon the circumstances of death. The 
secondary repression, symptomatized both by amnesia for re- 
lated events, and, in this case, by the phobia of bell-towers, 
protected the girl from stimuli which might arouse the pri- 
mary memories. 

Sears’ account of repression follows a procedure similar to 
that used by Hull in deducing other forms of behavior from 
conditioning. The main point is that any stimuli associated 
with repressive sequences tend, by conditioning, to evoke 
avoidance behavior. Hence recall is interrupted by avoid- 
ance, and the patient appears amnesic for the events which 
the cues would normally reinstate. A partial experimental con- 
firmation of his theory was reported by R. R. Sears (1937): 
This explanation of amnesia has much in common with the 
suggestions made in Chapter 10 (pp. 245f.) with regard to 
avoidance behavior. If it be assumed that tensions are 
aroused in the presence of cues which lead almost to the 
arousal of the repressed painful experience (i.e., cues which 
arouse rg’s), then the diversion of the behavior sequence to 
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neutral acts will be tension-relieving. Thus the avoidance be- 
havior is strengthened through tension-reduction without a 
recurrence of the original unpleasant memories. ; 

Guthrie’s (1938) theory of neurotic symptoms is based 
largely on the proposition that neurotic behavior never gets 
extinguished because the neurotic individual avoids the stim- 
uli in the presence of which extinction might take place. Thus 
fear of high places can never be overcome if the person with 
that fear always avoids high places. The avoidance sequence 
which in Sears’ theory is given a positive explanation, is 
in Guthrie’s given a negative explanation in terms of non- 
extinction. 

At the present stage of experimentation and theory, it is 
probably unwise to expect a comprehensive coordination be- 
tween the rich data of psychoanalysis and the restricted data 
of conditioning. Partial harmonization or coordination, such 
as that attempted by Sears, is probably more fruitful than 
more ambitious attempts to account for the whole range of 
clinical findings. 


Conditioned Emotions 


The story of Albert and the rat as reported by Watson and 
Rayner (1920) is a classic in conditioning literature. A rat 
was shown to the child at the same time that a loud sound 
was produced by striking a steel bar. The sound caused the 
child to cry. After a few repetitions, the child cried at the 
sight of the rat. This fear was generalized to other furry ob- 
jects, such as to his mother’s fur neckpiece. This single case 
study has remained the prototype of discussions of the de- 
velopment of emotions, although later experimenters have 
been unsuccessful in duplicating it. English (1929) reported 
three cases of attempted emotional conditioning; Bregman 
(1934) reported fifteen. Their lack of success indicates that 
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the process is not as simple as the story of Albert suggests. 

While the Watsonian account of emotional conditioning has 
been convenient for pedagogical purposes, its contemporary 
importance is lessened by two developments since the case re- 
port on Albert and the rat. In the first place, the difficulties in 
securing clean-cut repetitions of the experiment have called 
attention to other factors in the social situation which are at 
least as important as contiguous stimuli in producing the af- 
fective transference to novel objects. In the second place, a 
greater interest in maturation now directs attention to other 
ways than conditioning by which a few primitive emotions 
might become differentiated as a child grows older. 

The limitations of direct analogy from the conditioning ex- 
periment to the acquisition of emotional responses are evident 
enough if one thinks of commonplace situations in which such 
learning occurs. A child is playing in a room with its mother. 
The mother spies a mouse and utters a frightened shriek. Ac- 
cording to the pattern of the Albert experiment, the child, 
being frightened by loud noises, is frightened by the mouse, 
which is present while the noise occurs. If the simple associa- 
tion principle is to be applied, and the noise is the source of 
the fear, why does the child not fear the mother, who is more 
intimately associated with the shriek than the mouse? As a 
matter of fact, the child runs to the mother for protection. In 
order to be applicable, the conditioning explanation has to be 
supplemented by allowing for a number of additional facts. 
The mother has prestige, and the object of the mother’s fear 
is the object of the child’s fear. But this is not according to 
the conditioning prototype. It does not follow because fears 
are learned that simple conditioning is a sufficient explanation. 
The forms of emotion which are socially important, such as 
anxieties, feelings of guilt, loyalties, are developed in a social 
context with which conditioning has never adequately dealt. 
What service conditioning could render by emphasizing the 
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learned nature of emotions has already been given. The next 
stage requires a much more detailed treatment of the social- 
ization of the emotional life. 

The possibility that emotions are modified by growth as 
well as through specific experienced associations has been 
examined by a number of investigators. H. E. and M. C. Jones 
(1928) studied the fear responses of children and adults to a. 
harmless snake. Children up to the age of two showed no fear 
of the snake; definite avoidance responses emerged gradually, 
and were most pronounced in adults. The interpretation is 
not that adults had been more often frightened by snakes, but 
that their general perceptual and intellectual development 
contributed to their fear in a situation involving potential 
danger. Goodenough (1932) described a somewhat dramatic 
instance of the possibility of maturational factors in emotional 
expression. A ten-year-old girl who was totally blind and 
deaf from birth showed emotions by facial expressions and 
gestures similar to those of seeing and hearing children, al- 
though she obviously had not learned them through observa- 
tional experience. The increasing specificity and sequential 
differentiation of emotional patterns with age, similar to other 
developmental items, is reported by Bridges (1932). 

The chief service of conditioning to the mental hygiene of 
emotions has been its emphasis upon the acquired character 
of many affective manifestations. It was perhaps wholesome 
for a time to overemphasize the learned nature of fears and 
attitudes, many of which had earlier been thought to be in- 
stinctive. It is equally important now, however, to correct 
earlier exaggerations in the light of accumulated knowledge. 
It is not sufficient to attribute to conditioning all specific emo- 
tional attachments which have been acquired in the lifetime 
of the individual. Because hysterical and other neurotic symp- 
toms represent bizarre and unreasonable learning, apparently 
often as a result of chance association, conditioning analogies 
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appear to be appropriate. However, until the precise forms 
of phobias, compulsions, and other symptoms have been com- 
pared and coordinated with the data derived from conditioned 
response experiments, analogies must be resorted to very 
cautiously. 

Closely related to the conditioning of major emotions is the 
conditioning of preferences and attitudes of approval and 
disapproval. Several suggestions along these lines have been 
made recently by Razran (1938a, 1938b), who found that 
preferences for modern music could be increased by playing 
the music while the subjects were eating a free meal. Similarly, 
preferences for names or faces (photographs) could be mod- 
ified. The results are not yet available in complete enough 
form to be valued as more than interesting suggestions. 


Psychotherapy 


Conditioned response techniques have been used success- 
fully for many years in both the diagnosis and the treatment 
of psychoneurotic conditions. 

The detection of sensory function in simulated and hyster- 
ical anesthesia is a natural application of conditioning tech- 
niques, because sensory thresholds are measured without de- 
pendence on the verbal report of the subject. Bekhterev early 
reported his success with hysterical deafness (1912), and his 
procedure was confirmed later by Myasishchev (1929). R. R. 
Sears and Cohen (1933) studied a case of hysterical anes- 
thesia, analgesia, and astereognosis by conditioning methods. 
They found that when conditioned responses were established 
to stimulation of supposedly anesthetic areas the anesthesia 
disappeared. Cohen, Hilgard, and Wendt (1933) were able to 
demonstrate visual sensitivity in a patient with hysterical 
blindness, but the symptoms did not disappear as a result of 
conditioning. Various methods for detecting functional dis- 
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orders are available, and there is at present no convincing 
evidence that conditioning works where other methods fail. 

In addition to diagnosis, conditioning may be used in ther- 
apy. The following case, previously unreported, is described 
in some detail to illustrate the kind of clinical application 
which is feasible. 


Miss H., an unmarried school-teacher of 32, had been in 
a motor car accident six years before coming for treatment. 
During these six years her left arm had been entirely useless 
to her, being totally paralyzed and anesthetic. Neurological 
examination showed the anesthesia to be of the glove type, 
with a line of demarcation at the shoulder. The symptoms 
were thus shown to be functional. The arm and hand showed 
considerable atrophy from disuse. Attempts by a psychiatrist 
to give the patient insight into the functional nature of the ab- 
normality met with great resistance, although the fact that 
the patient had returned to the hospital voluntarily in the 
hope of a cure was considered a hopeful sign. The psychiatrist 
arranged to have the patient treated by conditioning methods 
in the psychological laboratory which adjoined the psychiatric 
unit. 

A finger-withdrawal experiment was arranged. Two elec- 
trodes were used, one for each hand. The first series of experi- 
ments consisted in presenting a shock to the anesthetic hand 
as the conditioned stimulus, a shock to the normal hand as 
the unconditioned stimulus. This was designed to give evi- 
dence of sensitivity in the anesthetic hand, since a shock to 
it served as the signal for withdrawal of the normal hand. 
While little conditioning occurred, the desired effect was pro- 
duced, and sensitivity gradually returned in the anesthetic 
hand and arm. Experiments were repeated daily, and the im- 
provement was gradual. After recovery of sensitivity, the 
conditioning procedure was reversed for purposes of develop- 
ing voluntary control. The normal hand was given a light 
shock which served as the conditioned stimulus. For the un- 
conditioned stimulus, the paralyzed hand was given a more 
severe shock, to which it was now fully sensitive. Presently 
movement began to occur in the paralyzed hand at the signal 
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given to the normal hand. This was the beginning of control, 
and voluntary movement was gradually restored. At this stage, 
the patient was given physiotherapy to strengthen the muscles 
which had been so long unused. The symptoms had not re- 


turned two years later, nor were any additional symptoms re- 
ported. 


This case illustrates both the feasibility of conditioning pro- 
cedures in the treatment of psychoneurotic symptoms, and 
the difficulty of interpretation. The experimental situation was 
one that permitted recovery without loss of face on the part 
of the patient. She believed the machine to be making the 
cure; in fact, the patient was so pleased with the contrivance 
that she wished to purchase it for home use! While the con- 
ditioning procedure was the effective means for bringing the 
Symptoms to an end, other dynamic factors in the situation 
were probably of great importance. 

A word of precaution with respect to the use of conditioning 
as a means of psychotherapy is probably in order. To use a 
method like that in the case above except under supervision by 
someone experienced in psychotherapy would be to run the 
risk of merely treating symptoms. Conditioning, like sugges- 
tion, may occasionally be useful, but its use is to be regulated 
in relation to all the factors in the case history as they bear 
upon the present symptoms. 

Conditioning offers some possibilities as an alternative to 
suggestive therapy in cases of drug addiction and alcoholism. 
Remarkable success with alcoholics has been reported by Kan- 
torovich (1929). Twenty alcoholics were treated by a condi- 
tioning procedure, while 10 others were treated by medication 
and hypnosis. The method used in conditioned therapy was 
the presentation of exposure cards with the words ‘vodka,’ 
‘alcohol,’ ‘drunk,’ followed by severe shocks. Also included 
were bottles of liquor, the odor of alcohol, the injection of 
alcohol, each followed by shock. Interspersed were other 
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words like ‘health,’ ‘work,’ ‘sober,’ and stimuli such as the 
odor of oil of cloves and the injection of physiological salt 
solution. These were not followed by shocks. Follow-up was 
reported on 17 of those treated by conditioning. Of these, 14 
refrained from drinking for from one to several months after 
treatment, while 3 were drunk within a few days. Of the con- 
trol group, only 3 refrained from drinking for from 2 to 6 
weeks, while 7 of the 10 got drunk within a few days. Without 
more information about the cases, the results can be accepted 
only as interesting illustrations of the way in which a modified 
conditioning procedure might be tried out. 


Individual Differences 


Anyone considering personality problems is confronted 
with evidence that individual differences exist between or- 
ganisms of the same age and species. Pavlov noted that some 
dogs appeared to be more excitable than others, and, contrari- 
wise, some appeared to be more inhibitable. He lent the 
weight of his authority to a classification of dogs according to 
types on the basis of their reactions in his experiments. The 
types were not considered to be rigidly distinct, but as points 
on a continuous scale, many dogs representing intermediate 
types. The differences between dogs were coordinated by Pav- 
lov with the ancient classification of temperaments attributed 
to Hippocrates. The extremely excitable dog was classified 
as choleric, the extremely inhibitable as melancholic. The 
central groups included the sanguine and the phlegmatic, 
with tendencies toward excitation and inhibition, respectively. 
The scaling is essentially one based on inhibition at one end 
and excitation at the other. 

Just what Pavlov meant by the temperamental differences 
between his dogs is best brought out through descriptive sum- 
Maries selected from his books. The following statements 
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agree closely with sentences from the books, but they have 
been assembled from different places and rearranged in order 
to gain continuity. 


The excitable type of dog (choleric) develops positive con- 
ditioned responses with ease. The responses reach their maxi- 
mum quickly, and persistently remain at this strength. They 
remain steady in spite of attempted extinction. Inhibitory 
conditioned responses develop, if at all, with great difficulty. 
If inhibitory and excitatory stimuli follow too closely together, 
the conflict results in a neurotic condition in which only excita- 
tory processes are retained, while inhibitory ones disappear 
almost completely. The condition is likened by Pavlov to 
neurasthenia. Bromides and calcium salts are useful in 
therapy. 

The central type of dog represents a normal balance of ex- 
citation and inhibition. There are two sub-types, one leaning 
toward excitation (the lively type), and the other toward in- 
hibition (the stolid type). 

_ The lively type (sanguine) is vivacious, alert, demonstra- 
tive. Varying stimuli are continually necessary. This type 
normally seeks such stimuli and is capable in their presence 
of expressing great energy. With monotonous stimuli, how- 
ever, it quickly and easily falls into a state of drowsiness an 

sleep. It is necessary to develop a variety of conditioned re- 
sponses concurrently, both excitatory and inhibitory, in order 
to keep an animal of this type alert in the experimental situa- 
tion. If handled in this way, it is a satisfactory experimental 
animal. 

_ The stolid type (phlegmatic) is normally quiet, peculiarly 
indifferent to external happenings, but always on the alert. It 
does not enter into either friendly or antagonistic relations 
with anybody, even its master. Both positive and negative 
conditioned responses are extremely precise. It never shows 
drowsiness in the stand. Pavlov once aroused a dog of this 
kind by making extraordinary sounds with a toy trumpet 
while wearing a frightening animal mask. The dog then lost 
its usual restraint, barked determinedly, and attempted to 
attack him. á 


The inhibitable type (melancholic) is timid or cowardly. It 


PERSONALITY 301 


moves cautiously, with tail tucked in, and legs half bent. At 
the beginning conditioned responses are exceedingly difficult 
to form, but when the difficulties are overcome, the dog acts 
like a perfect machine. Inhibitory conditioned responses are 
very stable; positive ones easily vanish in the presence of 
unusual stimuli. Under strain, the excitatory processes are 
weakened, and the experimental neurosis takes a form likened 
by Pavlov to hysteria. No cure, other than a rest of five or six 
months, was found. 


Pavlov, in attempting to choose excitable dogs, would at 
first pick them by their characteristic behavior outside the 
experiment. When chosen in this manner, a dog selected as 
most excitable was often found to go to sleep in the stand. His 
classification would therefore be changed from ‘choleric’ to 
‘sanguine.’ No description of naturalistic behavior is avail- 
able by which to differentiate the type of excitable dog which 
goes to sleep in the stand from the type of excitable dog which 
never shows signs of inhibition. Razran (1935) in experiments 
with human subjects could not predict from one situation 
to another what ‘type’ his subject would belong to, and he 
made no effort to relate his types to other personality char- 
acteristics. 

W. T. James (1934b) found that he could differentiate ex- 
citable and inhibitable dogs on the basis of their responses 
in a discrimination experiment. Excitable dogs showed imme- 
diate generalization and did not develop stable discrimina- 
tions. The more inhibitable dogs did not respond at all to the 
negative stimulus. There appears to be some relationship to 
morphological form. The characteristic dog of the passive 
type is the basset hound, of the excitable type, the German 
shepherd and saluki (W. T. James, 1938). Breeding experi- 
Ments are now under way at the Cornell Anatomy Station, 
Lake Mohegan, N. Y., to determine with greater specificity 
the relationship between bodily form and behavior. 

Loucks (personal communication) has described some of 
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the relationships between behavior under the natural living 
conditions and in experimental neurosis. He does not believe 
that the naturalistic behavior and the laboratory behavior are 
very highly correlated. For example, a dog showing highly 
‘neurotic’ symptoms in the laboratory (whining, refusal to 
eat, etc.) may act outside the laboratory in much the same 
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SPECIMEN DISTRIBUTIONS or Ease or CONDITIONING 


The data for fish are from Stetter (1929), those for dog from Pavlov’s 
laboratory. Both of these distributions are taken from Razran (1935). The 
data for man are from A. A. Campbell and Hilgard (1936). The criteria of 
conditioning are not alike in all the experiments, so that phylogenetic 
comparisons are unwarranted. 


way that it has always acted. Under other circumstances, 
there may be convincing carry-overs. Thus Drabovitch and 
Weger (1937) found one dog which went into an inhibitable 
state, described as a state of rest, which lasted for a month. 
Pavlov’s failure to present statistical evidence for the rela- 
tive frequency of types is in part corrected by Razran’s sum- 
mary of the rate at which conditioned responses were first 
formed in 60 dogs from Pavlov’s laboratory. The distribution 
1s reproduced in Figure 36 together with results for other 
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organisms. It is evident that most dogs condition readily, 
with only a few resistant to conditioning. If readiness of con- 
ditioning is taken as a mark of excitation, the excitable types 
are apparently more prevalent than the inhibitable ones. 
There is no evidence whatever of multimodality. Other dis- 
tributions of conditioning, both in animals and man, while 
differing in skewness, ordinarily agree in having a single 
mode. 

Distributions based on ease of differentiation have not been 
presented. James’ work, cited above, suggests that these might 
be more significant for temperament than distributions based 
on rate of simple conditioning. 

Pavlov’s dimension of excitation-inhibition is, of course, 
not the only possible dimension by which dogs can be classi- 
fied. W. T. James (1936, 1939) has made an interesting use 
of a dominance-submission dimension in classifying the re- 
sponses of some of his dogs. Conditioning in a social situation 
is greatly influenced by the temperamental characteristics of 
a dog in relation to a second animal, toward which it may react 
under everyday conditions as dominant or as submissive. 
During the first stages of such a study it is natural to classify 
animals into groups as predominantly submissive or predom- 
inantly aggressive, but these ‘types’ must presently give way 
to a scale on which all dogs can be classified. Only after the 
dominant-submissive dimension of behavior has been coor- 
dinated with a sufficiently large variety of other behavior can 
such a trait be said to have much significance. This is a pro- 
gram left for the future. 

Whenever efforts have been made to secure intercorrela- 
tions of measures between two conditioning situations or 
between aspects of conditioning situations and other person- 
ality measurements, such as intelligence test scores, suggesti- 
bility measures, personality inventories, the results have in- 


304 CONDITIONING AND LEARNING 


variably yielded low or insignificant correlations. There is no 
type or trait theory of conditionability which is yet satisfac- 
torily established. 


Clinical Types 


Extreme forms of derangement in human beings have often 
served as a point of departure for personality study. Pavlov 
in his later years became very much interested in them, and 
one writer has even been so bold as to designate him “the 
psychiatrist of the future” (Liddell, 1936b). 

Starting from his conceptions of types of nervous systems, 
and of the neuroses resulting when the extreme types are sub- 
jected to strain, Pavlov arrived at an interpretation of both 
neuroses and psychoses. Neurasthenia he thought is the result 
of serious conflict in which excitation prevails. Its extreme 
form is the manic-depressive psychosis. It may be recalled 
that some dogs become extremely excitable (manic), while 
some excitable dogs go to sleep in the harness (depressed). 
Hysteria, on the other hand, is a neurosis resulting from con- 
flict in which inhibition prevails. The tendency for the normal 
unification of cortical activity to break down in hysteria led 
him to compare it with schizophrenia, which he interprets as 
an exaggerated instance of such cortical disintegration. Among 
Pavlov’s last papers were some attempted interpretations of 
obsessional neuroses and paranoia. These he explained as per- 
severative phenomena based on a pathological inertness of 
cortical cells. 

A number of investigators have studied the conditioned be- 
havior of psychopathic individuals. Inconsistencies in the find- 
ings make it difficult to give any general summary of their 
results. Excitable subjects have been reported to condition 
readily, and manic-depressive patients appear to be somewhat 
more readily conditioned than schizophrenics. This presuma- 
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bly is in gross agreement with Pavlov. Schizophrenics, how- 
ever, have been found to equal or exceed the normal in cer- 
tain experiments with the galvanic response (Shipley, 19342), 
although in a knee jerk experiment they were less reactive 
than either the normal subjects or the manic-depressives, 
and gave more negative responses (i.e., kicking backwards) 
(Pfaffmann and Schlosberg, 1936). This might be interpreted 
also as supporting Pavlov’s contention that schizophrenics 
represent an extreme form of the inhibitable type. These agree- 
ments become unimportant, however, as soon as it is recog- 
nized that the conditioned responses give simply a rather poor 
instance of the well-known clinical picture of excitement in 
the manics and of the negativism that marks schizophrenia. 
The more crucial evidence, that the nervous systems were of 
these types prior to the onset of the symptoms, cannot be se- 
cured from such experiments. 


Appraisal of Conditioning in Relation 
to Personality 


Interest in the study of personality centers both in per- 
sisting individual differences and in the ways in which person- 
ality changes. The process of conditioning is fundamentally 
a process of change, and hence it is relevant to explanations 
of emotional attachments, and, in certain instances, to ex- 
planations of symptom formation and elimination. The use- 
fulness of conditioning in relation to personality will be en- 
hanced to the extent that it actually provides a satisfactory 
system of learning. Up to now the value of conditioning as a 
method for revealing persisting idiosyncrasies has been ex- 
ploited because of Pavlov’s theory of types and his demon- 
stration of experimental neurosis. 

The evidence for experimental neurosis is dependable, al- 
though the relation of these experimental upsets to neurotic 
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behavior in man is not clear. The presence of conflict in the 
genesis of experimental neuroses suggests a relationship to the 
Freudian theory of neurosis, although there is little similarity 
in the designation of the conflicting tendencies. The general- 
ization from experimental findings to theories of hysteria, 
neurasthenia, schizophrenia, and manic-depressive psychosis, 
is obviously premature. To have discovered that experimental 
neuroses may be produced under standard conditions, and 
studied under laboratory controls, is an important contribu- 
tion which may have great significance in the future. Out- 
standing limitations of the conditioning technique for person- 
ality study result from the use of but one animal at a time, 
with motivations essentially physiological, whereas human 
personality is preeminently social, and its conflicts are pre- 


dominantly between the needs of a person and the demands 
of his culture. 


Notes 
. Drabovitch and Weger (1937), 
Experimental Neuroses Dworkin (Caso) O 


Although experimental neuroses 
were not discovered in Pavloy’s 
laboratory until 1914, Bekhterev 
Saw certain clinical possibilities for 
conditioning almost as soon as he 
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began working with it (1908, 1909). 
A prize for the detection of simu- 
lated deafness by a conditioning 
method was awarded to Bekhterev 
in 1912 (Bekhterev, 1932, p. 204). 
In America early interest in the 
relation of conditioning to mental 
disease and to mental hygiene is 
represented by Watson (1916b) 
and Burnham (1917). 

The following references present 
data bearing on experimental neu- 
rosis: in the dog, Babkin (1938), 


and O. D. Anderson (1934), Liddell, 
Sutherland, Parmenter, and Bayne 
(1936); in the pig, Curtis (1938), 
Liddell (1938); in the rat, Cook 
(1939b), Humphrey and Marcuse 
(1939), Maier (1939b), C. T. Mor- 
gan and J. D. Morgan (1939); in 
birds, Bayandurov (1932); 1n cats, 
Dworkin (1939), Dimmick, Ludlow, 
and Whiteman (1939); in man, 
Gantt (1936), Hamburger (1933) 
Ivanov-Smolensky (19274, 1927b), 
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Liddell (1936a, 1936b), Razran 
(1933b). The methods for produc- 
ing experimental neurosis have been 
reviewed by Cook (1939a). 

Among the more important at- 
tempts to relate psychoanalytic 
theory to conditioning may be men- 
tioned Drabovitch (1935), French 
(1933), Hardcastle (1935), Hull 
(1939a), Humphrey (1920, 1921), 
Ischlondsky (1930), Kubie (1934), 
O. H. Mowrer (1939, 1940), Schilder 
(1929, 1935, 1937), R. R. Sears 
(1936). 


Conditioned Emotions 


A comprehensive account of the 
experiments on emotional condition- 
ing is given by M. C. Jones (1933). 
The more important experiments on 
the learning and unlearning of emo- 
tions by techniques related to con- 
ditioning are those of Bregman 
(1934), English (1929), Gauger 
(1929), F. B. Holmes (1936), H. E. 
Jones (1931), M. C. Jones (19242, 
1924b), Moss (1924), Watson and 
Rayner (1920). 

Changes in preferences by condi- 
tioning are discussed by Khozak 
(1934), Kovsharova (1934). 


Psychotherapy 


The uses of conditioning in psy- 
chotherapy are discussed by Bekh- 
terev (1912, 1923a, 1923c, 1932), 
Delmas (1930), Burnham (1917, 
1921), Cohen, Hilgard, and Wendt 
(1933), Drabovitch (1931), Gantt 
(1938c), Golla (1921), Guthrie 
(1938), Harrington (1938), Liddell 
(1936b), Marinesco and Kreindler 
(1935), Myasishchev (1929), Pann- 
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horst (1932), Schaeffer (1936), 
R. R. Sears and Cohen (1933), 
Shaffer (1936), Tinel (1930). 

The unpublished case reported in 
the text was studied by Hilgard in 
collaboration with Dr. Stanley B. 
Lindley. 

Gesell’s (1938) review of the con- 
ditioned reflex in relation to the psy- 
chiatry of infancy lists 57 references. 
The most important previous review 
is that of Seham (1932). Among 
the more significant accounts are 
those by Gemelli (1937), Hamburg- 
er (1914, 1933), [vanov-Smolensky 
(1927c), Karger (1923), Krasno- 
gorski (1925, 1930, 1931, 1935), Pei- 
per (1934). 

Some of the considerations of spe- 
cial symptoms are the following: al- 
coholism, Ichok (1934), Kantoro- 
vich (1929); allergy, Morhardt 
(1930); anorexia, Rouquier and 
Michel (1934); enuresis, Krasno- 
gorski (1933a), J. J. B. Morgan and 
Witmer (1939), O. H. Mowrer and 
W. M. Mowrer (1938); anxiety, 
H. Harris (1934); hysteria, Gackeil 
(1928), Marinesco (1937), Mari- 
nesco and Kreindler (1935), R. R. 
Sears and Cohen (1933), Tinel 
(1930); morphine addiction, Ru- 
benstein (1931); narcolepsy, M. 
Levin (1934b); psychogenic depres- 
sion, Ivanov-Smolensky (1928); sex- 
ual aberrations, Bekhterev (19232), 
Kostyleff (1927), Max (1935), 
Meignant (1935); stuttering, 
Moore (1938). 


Individual Differences 


The descriptions of Pavlov’s types 
are summarized chiefly from Pav- 
lov, 1927, 285-300; 1928, 360-390. 
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The classification is by somewhat ar- 
bitrary criteria, and the classifica- 
tory scheme underwent a few chang- 
es during the last years of Pavlov’s 
life. Frolov (1937) states that three 
dimensions are needed for classify- 
ing a dog; strength-weakness (re- 
ferring to ability to tolerate strong 
conditioned stimuli); lability-inertia 
(degree of stereotypy); balance-un- 
balance (predominance of excitation 
or inhibition). The four main types 
described in the text persist, how- 
ever. [heir order of frequency, from 
most common to least common, is 
said to be melancholic, sanguine, 
choleric, phlegmatic. 

Individual differences in condition- 
ing have been coordinated to type 
theories by Belialev and Lukina 
(1930), Cowan and Foulke (1934), 
Frolov and Charitonov (1931), 
W. T. James (1934b, 1938), Mari- 
nesco and Kreindler (1935), Mari- 
nesco, Kreindler, and Coppelman 
(1935), McDougall (1929), Osipova 
(1926b), Pen (1933), Razran (1935, 
1939m), Rosenthal (1932), Timmer 
(1931), Petrova (1934-1937). 

Differences independent of type 
theories were studied by A. A. Camp- 
bell (1938, 1939), A. A. Campbell 
and Hilgard (1936), Darrow and 
Heath (1932), Mateer (1918), Osi- 
pova (1927), Razran (19332, 
1933b), Schlosberg (1932). Corre- 
lations of these differences with in- 
telligence have been reported by 
A. A. Campbell and Hilgard (1936) 
Mateer (1918), Osipova (1927) 


, 
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Schlosberg (1932); with suggesti- 
bility by Berreman and Hilgard 
(1936), Schlosberg (1932); with 
personality measures by Darrow and 
Heath (1932), Marinesco, Kreind- 
ler and Coppelman (1935), Schlos- 
berg (1932); with morphological 
form by W. T. James (1934), 
1938), Timmer (1931). 


Clinical Types 


Pavlov’s papers dealing with clin- 
ical types include: neuroses in gen- 
eral (1932a), hysteria (1933), ob- 
sessional neurosis and paranoja 
(1934), neurotic and psychotic 
symptoms (1935). His theory of 
schizophrenia is reported by Kasat- 
kin (1932). Pavlov during the last 
years of his life spent much time in 
the wards of mental hospitals mak- 
ing observations on which to ground 
his theories. : 

Data on the conditioning of clin- 
ical patients have been reported on 
epileptics by Franklin (1928), Lu- 
kina and Matusova (1929), Mar- 
tino (1933); on manic-depressive 
psychotic patients, Ivanov-Smolen- 
sky (1925), Pfaffmann and Schlos- 
berg (1936); on progressive paraly- 
sis, Kantorovich and Lukina (1926), 
Rabinowich (1932); on schizophre- 
nia, Bender and Schilder (193°); 
Guk (1934), Landkof (1938), Mr 
rolyubov and Ugol (1933), Pfaff- 
mann and Schlosberg (1936), Ship- 
ley (1934a); on senile psychosis, 
Tatarenko (1934). 


Chapter 13 


NEUROPHYSIOLOGICAL MECHANISM OF 
CONDITIONING 


AvLov’s study of conditioning was almost entirely directed 
P toward understanding the functioning of the central nerv- 
ous system. His theories were expressed in the terminology 
of neural physiology, and he considered the investigation of 
conditioned responses as the true physiology of the cerebral 
hemispheres. The extension of Pavlov’s work in America has 
consisted largely of attempts to formulate a theory of learning 
in behavioral terms, but many special studies bear on the 
relation between neural function and conditioning. 


Pavlov’s Theory of Cortical Function 


The fundamental conceptions in Pavlov’s physiological 
theory are those of excitation and inhibition, conceived as 
states or processes with their locus in the cerebral cortex. The 
theory holds that as a result of afferent stimulation an ex- 
citatory process is initiated at a definite point in the cortex 
from which it gradually spreads over the entire sensory area 
or “analyser,” diminishing in intensity with distance from the 
primary point. This phase of irradiation is followed by a 
recession or “concentration” of the process to the point of its 
initiation. The duration of the irradiation and concentration 
varies in different experiments from a few seconds to twenty 
minutes. On the assumption that the receptive surface of the 
body is geometrically projected upon the cortex, the progress 
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of irradiation and concentration can be charted by their effect 
upon conditioned responses elicited from adjoining points of 
the receptive surface. In this respect inhibition shows the same 
characteristics as excitation. 

As the cortical process subsides at any point, it is succeeded 
in many cases by the opposite process. Pavlov has called this 
after-effect “induction.” Negative induction refers to the in- 
tensification of inhibition under the influence of preceding 
excitation; positive induction is the intensification of excita- 
tion following inhibition. The combination of induction with 
irradiation and concentration gives the completed picture of 
cortical function. Excitation, aroused at a particular point, 
spreads over the cortical analyser in wave-like form, the crest 
of the wave representing the excitatory process at its maxi- 
mum. During the next phase there is a reversal leading to a 
concentration of the excitation at the starting point, and a 
heightened inhibition at surrounding points. The phase of 
concentration is said by Pavlov to endure some five times as 
long as the phase of irradiation. 

The mechanism of the formation of a simple conditioned 
response is explained by Pavlov as follows. The excitation 
initiated by a neutral stimulus at point A irradiates over the 
cortex, and will be concentrated at any other focus of excita- 
tion such as that aroused by an unconditioned stimulus. After 
a number of repetitions of the two stimuli, the excitation 
aroused by the neutral stimulus is drawn to the locus of the 
unconditioned stimulus in sufficient intensity to elicit the con- 
ditioned response. The direction of the drainage of excitation 
is from the weaker to the stronger or more dominant focus 
of excitation. The phenomenon of generalization follows from 
the fact that excitation at a point B aroused by a stimulus 
similar to the conditioned stimulus will likewise irradiate 
widely and be drawn into the excitable focus of the uncon- 
ditioned response, either directly or by way of point A. Con- 
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tinued training channels the pathway between point A and 
the response point, and the gradual lowering of the excitability 
of the cortex outside this pathway accounts for the progressive 
decrease in generalization. If point B is extinguished by pre- 
senting its stimulus without reinforcement, the process at that 
point is changed from excitation to inhibition, which also ir- 
radiates over the surrounding region of the cortex. The extent 
of irradiation from any point will be a function of the limita- 
tions imposed by induction from other positive and negative 
points in the cortex. In an animal which has been subject to 
numerous conditioning series, the cortex can be considered as 
a functional mosaic of points of excitation and of inhibition 
representing various conditioned responses, whose interaction 
is governed by the principles of irradiation, concentration, and 
induction. 

Other forms of behavior are interpreted by Pavlov accord- 
ing to the same principles. Sleep is considered the result of 
extreme irradiation of the inhibitory process over the entire 
cortex and into the lower brain regions. Hypnosis is a form of 
partial sleep, or less extensive irradiation of inhibition. Neu- 
rosis is regarded as a functional pathology of the cortex in 
which, as a result of an unusually acute clashing of the excit- 
atory and inhibitory processes, or of the influence of strong 
and extraordinary stimuli, there is an exaggerated predom- 
inance in the animal of either excitation or inhibition, anal- 
ogous respectively to the hysteria or neurasthenia observed 
in human patients. 

In spite of its tempting simplicity and scope, Pavlov’s 
conception of cortical physiology has not met with any 
wide degree of acceptance. Four primary objections may be 
raised to it. 

1. Concepts of cortical physiology should be based upon 
more direct measures of cortical function. Pavlov’s inhibition 
and excitation are purely inferential concepts derived from 
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measures of overt movements or of amount of saliva se- 
creted. Likewise irradiation, concentration, and induction are 
derived solely from the behavioral observations of generaliza- 
tion, discrimination, and aiter-effect. The concept of drainage 
is merely a figure of speech without any accepted neurophysio- 
logical basis. Inferential concepts are of course necessary in 
the systematization of the experimental data, but little is 
gained by asserting that such concepts denote cortical proc- 
esses unless specific verification can be obtained through 
more direct observation of the cortex. 

2. The temporal characteristic of irradiation, one of the 
most fundamental points in Pavlov’s systematization, does 
not rest upon adequate experimental verification. A careful 
analysis by Loucks of the original data of the experiments on 
irradiation indicates that there is no “significant evidence for 
a sluggish spreading of an inhibitory disturbance from one 
restricted region to surrounding areas” (1933, p- 44). The 
general validity of any simple statement of irradiation for 
all regions of the cortex must be seriously doubted in view 
of recent findings by Dusser de Barenne and W. S. McCul- 
loch (1938). Using electrical recording of neural activity in- 
duced by application of strychnine to a point on the cortex, 
they have shown that within certain areas (visual cortex and 
Brodmann’s area 5 of the somato-sensory cortex) there is al- 
most no spread of activity to surrounding cortex, while in 
other areas there is irradiation to widely distant regions. This 
irradiation, moreover, is not uniform in all directions. Stim- 
ulation of a point in one area activates a second area, whereas 
the second area may not activate the first. A given point may 
augment the spontaneous activity of one area and depress that 
of a second. 

_ 3. The spatial character of Pavlov’s conception of cortical 
irradiation limits its general application. The assumption of 


NEUROPHYSIOLOGICAL MECHANISM 313 


a geometric projection of the skin surface onto the cortex is 
particularly relevant in accounting for the generalization of 
a conditioned response from one cutaneous point to a neigh- 
boring point. This explanation is not applicable to generaliza- 
tion among stimuli differing in intensity or quality which are 
applied to the same cutaneous point. In these cases the dis- 
tance of cortical irradiation cannot determine the degree of 
generalization. The difficulty becomes clear in considering 
generalization in the intensity dimension, where the neural 
excitation may be mediated by identical fibers but at different 
frequency. All discrimination may perhaps depend on some 
sort of dynamic separation of areas of excitation, as suggested 
in the Gestalt concept of isomorphism (Kohler, 1929), but this 
remains to be empirically demonstrated. Even the theory of 
isomorphism does not imply simple areal representation as 
proposed by Pavlov. 

4. Pavlov’s physiological conceptions are explicitly based 
upon the premise that conditioning is exclusively a cortical 
function. Recent experimentation which will be summarized 
later demonstrates, however, that conditioning is possible at 
a subcortical level. Although the importance of the cortex in 
learning by higher mammals is unquestionable, any theory 
that neglects the functioning of lower levels of the nervous sys- 
tem must be inadequate. The two-dimensional character of 
Pavlov’s irradiation concept does not easily permit extension of 
the theory to embrace the integrated functioning of cortical 
and sub-cortical centers. 


Anatomical Localization of Conditioning 


Experimental investigation of the neurophysiological mech- 
anism of conditioning has not been directed toward the ex- 
amination of Pavlov’s theories, but instead has been almost 
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entirely limited to the problem of the locus within the cen- 
tral nervous system of the modification responsible for con- 
ditioning. 

Pavlov believed that conditioning was impossible in the 
absence of the cerebral cortex. Subsequent work, however, 
has forced a modification of this view. His judgment was based 
upon the failure of Zeliony (1929) in 1912 to establish con- 
ditioned responses in dogs after surgical removal of the cere- 
bral hemispheres. Later, Poltyrev and Zeliony (1930) were 
able to establish motor conditioned responses to visual and 
auditory stimuli in three decorticate dogs, and Lebedinskaia 
and Rosenthal (1935) were successful in establishing a con- 
ditioned salivary response to a metronome. In all of the ani- 
mals the operations left a small remnant of cortex at the base 
of the brain, and so great was the influence of Pavlov’s opinion 
that in both studies the conditioning was attributed to the 
residual cortical tissue. Clear evidence of subcortical condi- 
tioning, however, has been presented by Ten Cate (1934c), 
who established a conditioned response to a bell in a totally de- 
corticate cat, and by Culler and Mettler (1934), Girden, Mett- 
ler, Finch, and Culler (1936), Poltyrev (1936), Zeliony and 
Kadykov (1938), who trained decorticate dogs in conditioned 
responses to optic, cutaneous, and auditory stimuli based on 
unconditioned electric shock to the foreleg. Incidental obser- 
vation of learned responses have been reported in other studies 
of decorticate cats and dogs (Rothmann, 1923; Schaltenbrand 
and Cobb, 1931; Dusser de Barenne, 1919; Bard, 1934)- 

The possibilities of subcortical conditioning can perhaps be 
better evaluated from experiments which involve partial re- 
moval of the cerebral cortex. Pavlov’s students and many 
subsequent investigators have shown that bilateral removal 
of a cortical sensory area does not prevent the establishment 
of conditioned responses to stimuli of that sense modality. 
Pavlov believed that such conditioning was mediated vicari- 
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ously by the remaining cortical tissue. Two lines of evidence 
render this interpretation highly improbable: (1) Condition- 
ing has been demonstrated after complete removal of the cere- 
bral hemispheres. Similarly, destruction of the sensory areas 
which removes the only direct anatomical pathway by which 
impulses from the sense organ reach the remaining cortex, 
does not prevent conditioning. (2) Conditioned responses es- 
tablished prior to cortical operation can be retained after op- 
eration without any further training (D. G. Marquis and 
Hilgard, 1936). Direct test of the vicariation hypothesis has 
been made by Lashley (1922) using an instrumental response 
to brightness in rats, and by Ten Cate (1938) using condi- 
tioned leg retraction to light in dogs. The learned response, 
which survived bilateral removal of the occipital lobes, was 
found to be unaffected by removal of any portion of the re- 
maining cortex. The only satisfactory interpretation of these 
studies is that conditioning does not necessarily depend upon 
cortical functioning. When the cortical projection area which 
normally participates in the response is lacking, the function is 
not taken over vicariously by other cortical areas, but is pre- 
sumably mediated by subcortical association centers, together 
with remaining non-specific cortex which may have partici- 
pated in the learning of the original act. 

The cortical motor center for the unconditioned response 
has likewise been found not to be essential for the formation of 
conditioned responses. Conditioned salivation is possible after 
removal of the area controlling salivation; conditioned blink- 
ing is unimpaired by removal of the motor eye area. The 
results obtained in attempts to establish conditioned leg with- 
drawal responses in dogs after destruction of the cortical sen- 
sorimotor region are still a subject of controversy. 

The idea of an association center in the cortex—a region 
involved in all learned responses irrespective of stimulus mo- 
dality—was common in the speculation of early workers. The 
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frontal lobes, having no other demonstrable function, have 
often been assigned an association function. Experimental 
investigation, however, has revealed no impairment in condi- 
tioned responses or simple instrumental discriminations fol- 
lowing removal of the frontal lobes in dogs (Pavlov, 1927; 
Bekhterev, 1923b; D. G. Marquis and Hilgard, 1936; Ten 
Cate, 1938) or in monkeys (Franz, 1907; Jacobsen, 1936). 

The possibilities of conditioning at the subcortical level are 
definitely limited, at least in the dog and higher mammals. 
After complete decortication, conditioned responses can be 
obtained only with intense stimuli and after long periods of 
training. Moreover, only the simplest forms of discrimination 
have ever been demonstrated. Culler and Mettler (1934) used 
a training method which permitted the dog to escape the 
shock to the paw by a prompt withdrawal response following 
the conditioned stimulus. In normal dogs the first few condi- 
tioned responses involved widespread activity such as strug- 
gling and yelping and resembled the unconditioned response 
to shock. With further training the response became restricted 
to a precise adaptive leg flexion. The decorticate dog showed 
only the former type of conditioned response. The possibility 
that decortication prevents the establishment of instrumental 
avoidance responses while still permitting conditioning by 
substitution deserves further investigation. 

After removal of the sensory areas the possibilities for con- 
ditioning are greater than after complete removal of the 
cortex. Quantitative study of the characteristics of conditioned 
lid closure to light in dogs and monkeys demonstrated that 
removal of the visual cortex produced almost no change (D. G. 
Marquis and Hilgard, 1936, 1937). The rate of acquisition 
and extinction of the response did not differ in the operated 
animals from the normal. The form of the response was un- 
altered, although its latency was slightly increased. Simple 
instrumental discrimination of light intensity was also unm- 


NEUROPHYSIOLOGICAL MECHANISM 317 


paired, and the differential threshold was only very slightly 
raised (D. G. Marquis, 1934; Kliiver, 1936; K. U. Smith, 
1937). Discrimination of visual shape, size, color, and position, 
however, seems to be impossible. The difference between the 
residual visual ability of decorticate animals and of animals 
with only the visual cortex removed cannot be explained on 
the basis of present evidence. It seems likely that the greater 
deficiency of the former is due to the impairment of non- 
visual functions involved in the performance of the particular 
conditioned response. 

The part played by the cortex in learning can only be ade- 
quately understood in the light of the evolution of the nerv- 
ous system. In submammalian vertebrates there is no true 
neocortex, and learning is mediated by striatal, thalamic, and 
tectal centers of the brain stem. Within the mammalian series, 
as the cortex becomes relatively larger and more highly dif- 
ferentiated, certain functions originally localized within lower 
centers of the brain are mediated by the cerebral hemispheres. 
This functional encephalization has been recognized by anato- 
mists and physiologists as the underlying principle of the 
evolutionary development of the central nervous system. Con- 
ditioned responses may be expected to show a greater de- 
pendence upon the integrity of the cortex in the dog as com- 
pared with the rabbit, and less in the dog than in the monkey 
and man. Although the available evidence is far from com- 
plete, much of it is in line with this view (D. G. Marquis, 
1934, 1935; Harlow, 1936). 


Functional Components of the 
Conditioning Mechanism 


The neurological mechanism of the association formed in a 
substitutive conditioned response may be considered in terms 
of the components of the afferent-efferent system which are 
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susceptible to separate experimental study. The basic prob- 
lem is to determine the locus of the association by which the 
conditioned stimulus is able to evoke the unconditioned re- 
sponse or some fraction of it. This association could logically 
be formed at any point at which the conditioned and uncon- 
ditioned pathways converge. Without specifying the neu- 
rological level, which might be cortical, subcortical, or periph- 
eral, we can list the possible loci of the association as lying 
in (1) the pathway activated by the conditioned stimulus, 
(2) the pathway activated by the unconditioned stimulus, 
(3) the motor pathway of the response to the unconditioned 
stimulus, or (4) an association pathway lying outside any 
of the three named above. These components and their pos- 
sible interrelations are represented diagrammatically in Figure 
37- This diagram does not denote anatomical regions or 
neurons, but merely represents the functional components 
which must necessarily be considered in any analysis of the 
mechanism of conditioning. Before such an analysis can be 
carried out in neuroanatomical detail it must be pushed farther 
on a behavioral level. 

Pavlov’s conception of the mechanism of conditioning im- 
plies that the association is formed at M, where the excitation 
irradiating from CS is concentrated. Razran (1930) points 
out that the locus of modification must lie in the uncondi- 
tioned rather than in the conditioned pathway because of the 
greater generalization which occurs among conditioned re- 
sponses involving the same response than among those hav- 
ing the same stimulus but different responses. Beritov (1924) 
believes that the foci of the conditioned stimulus, the uncon- 
ditioned stimulus, and the unconditioned response all have 
an increased excitability. The modifications in US play the 
most important part in the early stages of conditioning, while 
direct association of CS and M occurs in the well-established 
conditioned response. The evidence bearing on the role of 
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each of the components in the process of conditioning will be 


presented below. 

The Pathway of the Conditioned Stimulus. Complete 
interruption of the afferent pathway of the conditioned stimu- 
lus will necessarily prevent the formation or performance 


Conditioned Original response to 
stimulus ( : ) ene Stimulus 





Unconditioned 
Stimulus 


x 
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SCHEMATIC DIAGRAM OF THE PossIBLE RELATIONS AMONG THE COMPONENTS 
ĪNVOLVED IN SUBSTITUTION CONDITIONING 


There are four possible points at which the convergence of the pathways 
would logically permit the formation of the association implied in condition- 
ing: CS—the pathway activated by the conditioned stimulus, US—the 
pathway activated by the unconditioned stimulus, @M—the motor pathway 
of the response to the unconditioned stimulus, 4—a hypothetical association 
Pathway outside the three named above. 


FIGURE 37 


of the conditioned response. Sense organ stimulation, how- 
ever, is not necessary, and may be replaced by electrical 
Stimulation at any point on the sensory pathway. For this 
purpose Loucks (1934) has developed a method for stim- 
ulation of the nervous system in intact unanesthetized ani- 
mals. A paraffin covered coil of wire is embedded under the 
skin of the animal and insulated wires are led from it to the 
Particular point to be stimulated. No wires penetrate the 
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skin, and the animal remains in normal health for unlimited 
periods of time. The buried coil can then be activated by 
induction from a primary coil external to the animal. With 
this method successful conditioning has been produced by sub- 
stituting electrical stimulation of the spinal cord, cerebellum, 
(Gantt, 1937) or cortical sensory area (area striata) (Loucks, 
1938b) for the conditioned stimulus. 

Incidental evidence concerning the functioning of the cen- 
tral representation of the conditioned stimulus has been ob- 
tained from observation of hysterical and hypnotic sensory 
changes. A conditioned stimulus is rendered ineffective if the 
subject is hysterically anesthetic (Bekhterev, 1912), but pro- 
longed training may break down the anesthesia (R. R. Sears 
and Cohen, 1933). Under hypnotic anesthesia the conditioned 
stimulus is likewise ineffective (Lundholm, 1928; Erickson, 
1938b). Lundholm (1928) has shown in addition that a hal- 
lucinatory sensation can serve as a conditioned stimulus (cf. 
pp. 2orf.). 

A direct demonstration of the participation of the cortical 
sensory area in the conditioning modification has been re- 
ported by Martino and Fulchignoni (1938). They used strych- 
nine, which when applied locally to a neural center causes an 
abnormal overactivity of the nerve cells. A dog was trained to 
give a conditioned blink response of the left eyelid to red 
light, and of the right eyelid to violet light. (The effective dif- 
ference in the two stimuli was probably one of intensity since 
dogs are color blind.) When strychnine was applied to the 
visual center in the right occipital lobe, red light would pro- 
voke clonus of the left eyelid, followed by generalized epilep- 
toid seizures first appearing in that muscle. Violet light had 
no effect. If the strychnine was applied to the left occipital 
lobe, violet light produced the same results in the right eyelid, 
while red light was ineffective. Presumably, the conditioning 
of the left eyelid to red light had “opened” a pathway from the 
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right occipital visual center to the motor center in the right 
hemisphere controlling the eyelid on the left. 

The Pathway of the Unconditioned Stimulus. The. 
sense organ and afferent pathway of the unconditioned stim- 
ulus are not essential in the establishment of the conditioned 
response as long as excitation reaches some central point and 
evokes an unconditioned response. Successful conditioning 
has been produced by substituting for the unconditioned stim- 
ulus electrical excitation of lumbar posterior roots or dorsal 
columns of the spinal cord (Loucks and Gantt, 1938), and 
of the lateral cerebellar lobe (Brogden and Gantt, 1937). In 
stimulating the spinal cord a moderate strength which evoked 
leg movement was not adequate for conditioning; stronger 
stimulation, however, was effective. The difference is attrib- 
uted by the authors to the excitation of pain fibers in the 
latter case. 

An early experiment by Lang and Olmsted (1923) raised 
the question whether or not the sensory pathway of the un- 
conditioned stimulus must be intact for the performance of 
an established conditioned response. In their experiment a 
conditioned leg withdrawal to a buzzer accompanied by a 
shock to the leg was established in dogs. Hemisection of the 
spinal cord on the side opposite the trained leg, interrupting 
the ascending pathway of pain fibers, abolished the foot with- 
drawal response to the buzzer although the respiratory re- 
sponse remained. More recently Martino (1939) has reported 
that the conditioned blink response in dogs is abolished by 
anesthetization of the skin around the eye—the receptive 
field of the unconditioned response. He concluded that a 
constant stream of neural impulses from the unconditioned 
receptor must be present to summate with the excitation 
from the conditioned stimulus in evoking the conditioned 
response. Other investigators (Raeva and Rappoport, 1934; 
Yushchenko, Rolle, and Pupko, 1934; Settlage and Harlow, 
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1936), however, have failed to find any disruption of the 
conditioned response resulting from interruption of the path- 
way of the unconditioned stimulus. The divergent results 
remain unexplained. Harlow (1937) has shown that a con- 
ditioned response is not abolished by functional inactivation 
of the unconditioned pathway through habituation, indicat- 
ing that the central as well as the peripheral pathway of the 
unconditioned response need not be active during the per- 
formance of a conditioned response. 

The Unconditioned Response. Conditioning does not 
occur if the unconditioned response is directly elicited by 
stimulation of the motor pathway, but only if the excitation 
is initiated reflexly; i.e., by way of the central nervous system. 
If a conditioned stimulus is repeatedly followed by electrical 
stimulation of a motor nerve evoking muscular movement, 
conditioning is unsuccessful (Tracy, 1927; Hilgard and M. K. 
Allen, 1938), although some modification of behavior will 
occur in human subjects through voluntary facilitation of 
reactions secondarily aroused by the twitch (Yacorzynski 
and Guthrie, 1937). Loucks (1935) has shown that if an un- 
conditioned leg response is evoked by stimulation of the 
motor cortex in dogs (with the buried coil technique), no 
conditioning results in hundreds of trials. The nervous system 
is not impaired in any way by the procedure, for instrumental 
responses are readily formed by giving the dog food after the 
leg movement. 

Similarly, if the unconditioned response is produced by 
drugs which act directly on the response mechanism, no Con- 
ditioning is possible. The injection of pilocarpine causes añ 
increased secretion of saliva, but no amount of pairing with 
a neutral stimulus will lead to conditioned salivation (Kleit- 
man, 1927; Finch, 1938b). On the other hand, conditioned 
Salivation is readily established by the injection of morphine 
(Kleitman and Crisler, 1927). The difference lies in the fact 
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that morphine acts upon neural centers to excite the nerves 
to the salivary glands, whereas pilocarpine acts peripherally 
upon the gland cells themselves. For the same reason failure 
is encountered in the attempt to establish conditioned hyper- 
glycemia by injection of epinephrine (Gantt, Katzenelbogen, 
and Loucks, 1937; Loucks, 1937a), and conditioned gastric 
secretion by injection of histamine (Katzenelbogen, Loucks, 
and Gantt, 1939). These responses are aroused by peripheral 
action of the drugs. Other responses, such as fear, which are 
aroused at the same time through reflex pathways by the in- 
jection procedure, are susceptible to conditioning. 

Further evidence on the rôle of the response pathway is 
derived from studies in which the unconditioned response is 
prevented from occurring during conditioning. This preven- 
tion must be accomplished by some means which does not 
impair the other components of the conditioning mechanism 
or the results are not conclusive. Crisler (1930) was able to 
establish conditioned salivation in dogs based on the uncon- 
ditioned response to morphine even though the actual salivary 
secretion was prevented throughout the training period by 
paralyzing the glands with an injection of atropine. Similar 
results were obtained by Finch (1938c), using salivation to 
acid in the mouth as the unconditioned response. Light and 
Gantt (1936) paralyzed the right hind leg in four dogs by 
crushing the ventral motor roots, and trained the dogs by re- 
peated combination of an auditory signal with a shock to the 
paralyzed leg. A generalized conditioned response developed 
which involved defensive reactions of other parts of the body 
but no movement of the paralyzed leg. Training was discon- 
tinued while the crushed nerves regenerated. After function 
was restored in the paralyzed limb, the conditioned stimulus 
was found to evoke leg withdrawal—the appropriate and 
specific conditioned movement, but one which had never been 
evoked during training. The interpretation of these results 
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is not clear because of the fact that the conditioned stimulus 
elicited a general conditioned response at all stages of the 
training, and the specific paralyzed response then usually oc- 
curred in the tests as a portion of a larger avoidance pattern. 
In one dog, however, the movement occurred in an isolated 
way without other defense movements. 

Complete motor paralysis was secured by Harlow and 
Stagner (1933) by using curare. Training in which a buzzer 
was associated with an electric shock failed to produce a con- 
ditioned motor response in animals tested after they had 
recovered from the curare paralysis. It might be objected 
that the drug depressed brain functions, but pupillary dilata- 
tion to the shock, which was not paralyzed by the drug, 
showed conditioning. Girden and Culler (1937) found that 
even in complete curare paralysis a small twitch of skeletal 
muscles can be reflexly evoked, and that this response can 
be conditioned. They suggest that curare depresses cortical 
functions which are necessary for adaptive motor condition- 
ing, and that the conditioning of the pupillary response and 
of the muscle twitch represent subcortical conditioning. Until 
the possible central effects of curare are better understood 
these experiments do not permit unequivocal interpretation 
(Culler, Coakley, Shurrager, and Ades, 1939; Harlow and 
Settlage, 1939). 

An Association Pathway. The available evidence, al- 
though not conclusive, indicates that the locus of the condi- 
tioning association lies outside the regular pathways of the 
conditioned stimulus, the unconditioned stimulus, and the 
unconditioned response. The obvious alternative is the as- 
sumption of an association pathway, as represented by A 
in Figure 37. This does not, of course, imply the existence of 
an “association center” in the brain. Anatomical evidence rez 
viewed earlier rules out the possibility of a single region 1 
which associations are formed, but does not conflict with the 
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conception of an association pathway which is different for 
each combination of stimuli involved in the conditioning. 

The assumption of an association pathway fits the data 
concerning the changes in excitability of the component re- 
sponses. If the association were at CS it should be reflected in 
a greater excitability of the original response to the condi- 
tioned stimulus. This response, however, usually shows a pro- 
gressive decrement during conditioning (see Figure 7, p. 112). 
If the association were at US or at M an increased excitability 
of the response to the unconditioned stimulus would be ex- 
pected, while the typical result is a decrement in this response 
also. If an independent association pathway is postulated, the 
strength of the conditioned response can vary independently 
of the strength of the original responses to the conditioned 
and unconditioned stimuli. 

Studies of the effect of anesthetics and drugs indicate that 
conditioned responses are much more affected than are the 
original reflex responses to the conditioned and unconditioned 
stimuli, implying that the central pathways are not identical. 
Settlage (1936) has undertaken conditioning in cats which 
were given a dose of amytal sufficient to prevent the appear- 
ance of any conditioned responses but which did not depress 
the original unconditioned response to shock on the leg. Sub- 
sequent tests after recovery from the drug showed that a 
conditioned response had been successfully formed although 
it never appeared during the training. 

Cortical association pathways might be found to lie within 
the primary sensory areas, in surrounding areas, or at a dis- 
tance. Direct evidence of location in areas bordering on the 
primary area, but outside it, appears to have been obtained 
in an experiment by Culler, of which only a preliminary ab- 
stract has been published (1938a). Dogs were trained to 
withdraw the leg in response to a tone stimulus presented 
in combination with a shock to the paw. The response was 
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observed as a contraction in the exposed semi-tendinosus 
muscle of the leg. The cerebral hemispheres were then ex- 
posed by removing the overlying bone under suitable anes- 
thesia. Exploratory electrical stimulation of the cortex con- 
tralateral to the trained leg revealed a small spot, not over 2 
mm. square, located in the anterior ectosylvian gyrus, from 
which the conditioned leg response could be elicited. This spot 
was not in the primary auditory area nor was it in the somes- 
thetic sensory area; it was not the motor point for leg flexion, 
which was 20 to 25 mm. distant near the cruciate sulcus. Dogs 
tested prior to any conditioning showed no evidence of re- 
sponse to stimulation of the spot, and the response could be 
evoked only so long as the muscle responded to the condi- 
tioned tone stimulus. When the conditioned response was ex- 
tinguished by unreinforced repetitions of the tone, the re- 
sponse to electrical stimulation of the spot also disappeared. 
When the response to tone was reconditioned, the response to 
the spot could again be evoked. These observations indicate 
that the site of the association pathway may be very discretely 


localized, and open a vast field of study of the neural mechan- 
ism of conditioning. 


The Nature of Synaptic Modification 


The preceding discussion has been concerned with the gross 
localization of the events involved in conditioning, and the 
nature of the actual neural modifications may now be consid- 
ered. The problem is to understand how a stimulus which 
originally does not elicit a certain response, comes by virtue 
of the reinforcement procedure to evoke that response. Stated 
more generally, the essential alteration is a relatively perma- 
nent decrease in the threshold for evoking the response by 
the conditioned stimulus. This alteration occurs in the brain 
centers. Since conduction of the nerve impulse along a fiber 
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is believed to be “all-or-none” throughout its length, the 
changes in threshold must be localized at the point of excita- 
tion; namely, at the synaptic junctions between neurons. Àl- 
though the knowledge of synaptic transmission has been 
tremendously advanced in recent years, the experimental 
studies have been concerned exclusively with the immediate 
effects of stimulation. The excitatory process which gives rise 
to a propagated impulse has a very brief duration (0.2 
msec.) (Lorente de Nó, 1938a). It is followed by alterations 
in threshold, associated with negative and positive after- 
potentials, which may endure for 40 to 100 msec. (Eccles, 
1935). Theories of the more permanent modifications respon- 
sible for learning are necessarily speculative, and are based 
on extensions of current conceptions of neural excitation. Many 
of the earlier speculations have been invalidated by subse- 
quent experimental data. Some of the more plausible theories 
will be briefly summarized here. 

Anatomical Growth of Axons and Dendrites. Kap- 
pers (1917) has proposed that relatively permanent new con- 
nections between neurons are established by the growth of 
neural processes under the influence of bioelectric potentials 
resulting from stimulation. If two cells are simultaneously ex- 
cited, the resulting ionization is assumed to direct the growth 
of axons toward the cathode region, and of dendrites toward 
the anode, such that a new synaptic connection is established. 
This theory, which arose out of an attempt to account for 
changes in position of tracts and nuclei in phylogenetic de- 
velopment, has been used intensively by Holt (1931) who 
makes it the basis for a complete theory of learning. No direct 
experimental evidence exists either to support or to disprove 
the theory. It is not necessary to postulate that anatomical 
growth is essential for the establishment of new neural con- 
nections. Histological study demonstrates a very rich branch- 
ing and interlacing of neurons in the central nervous system, 
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and physiological observations of the spreading of response 
after administration of strychnine indicate that there is a po- 
tential anatomical connection between any afferent and any 
efferent neuron. 

Changes in the Physico-Chemical Properties of the 
Nerve Cell. Excitation of a neuron is believed to be a process 
of breaking down (depolarizing) the membrane at the point 
of the synaptic connection. Among the changes suggested as 
necessary for learning are changes in one or the other of these 
properties: a concentration of some substance within the cell, 
a reorientation of the molecules of the membrane, changes in 
the collodial dispersion within the cell, changes in surface ten- 
sion, changes in the interior-exterior potential gradient. There 
is no dependable knowledge relating these processes to learn- 
ing. 

The primary problem of the neurophysiology of learning, 
however, is to specify the conditions of this change, whatever 
it may be. This may be stated in its simplest form by refer- 
ence to Figure 38: starting with an anatomical but not a func- 
tional connection between fibers CS and a, how does simul- 
taneous stimulation of fibers CS and UncS alter the threshold 
of a so that subsequent stimulation of CS alone will activate 4 
and thus Runes? Watson (1914) and others have made the 
unqualified assumption that repeated activation of any path- 
way increases its excitability. Repetitive activity, however, 1$ 
the typical condition for adaptation, which presumably de- 
pends upon the decreased excitability of a synapse. Pavlov 
does not present any specific solution of this problem, but 
speaks of drainage or irradiation of “neural energy, without 
considering the properties of nerve excitation and conduction. 
Translated into the terms of Figure 38, his conception would 
assume that simultaneous excitation of fiber a by two axons 
somehow produces a long-lasting decrease in threshold. More 
explicit mechanisms have been suggested by Cason (1925) 
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and by Washburne (1935) in terms of ionic interchange, by 
Johnson (1927) in terms of a tuning of membrane film- 
frequencies, and by Marinesco and Kreindler (1935) in terms 
of equalization of chronaxie. The essential basis of all such 
theories is the simultaneous activation of a neuron from two 
sources. Recent experimental work (Lorente de Nó, 1938b) 
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Ficure 38 


SCHEMATIC ARRANGEMENT oF Neurons To Account FOR CONDITIONING BY 
ALTERATION OF SYNAPTIC THRESHOLD 

Impulses in neuron UncS, set up by the unconditioned stimulus, evoke a 
response in the neuron Runes, which participates in the unconditioned 
response. Impulses in CS excite neuron Reg (the original response to the 
conditioned stimulus) but are not effective at first in exciting a. If CS and 
UncS (and therefore a) are simultaneously activated, there occurs an increase 
in the excitability of a such that CS alone is eventually able to excite it. 

This diagram differs from conventional ones only by the addition of the 
fiber a. If the collateral of CS ended directly on Rones, the threshold change 
in the latter neuron would necessarily be accompanied by an increased 
excitability of the reflex response to the unconditioned stimulus. The present 
diagram permits the threshold of the conditioned response to vary inde- 
pendently of the original responses. 


has demonstrated that spatial summation of at least two 
simultaneous impulses is necessary to excite a neuron under 
any conditions. No explanation has yet been proposed why 
such excitation produces a permanent threshold change in 
some cases and not in others. 

Continued Activity in Closed Neural Chains. Recent 
observations indicate that the path of neural impulses from 
sensory to motor nerves is not a single “straight-through” 
pathway (Lorente de Nó, 1938c). Histological and experi- 
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mental data indicate that the internuncial neurons are ar- 
ranged in two types of chains: multiple chains, in which sev- 
eral collaterals of a single fiber, after traversing one or more 
synapses, converge upon a motor neuron, and closed chains, 
in which a collateral excites a circle composed of several 
neurons. In the latter case the chain of neurons may maintain 
its activity indefinitely in the absence of peripheral afferent 
impulses. This arrangement suggests another possible mechan- 
ism of learning which would not necessarily involve any 
permanent alteration in the physico-chemical properties of 
the neurons. Closed chains, set into activity by the training 
procedure and continuing in the absence of any external ex- 
citation, would summate with otherwise inadequate afferent 
impulses to produce the conditioned response (Figure 39). 
Rashevsky (1938) has presented a mathematical analysis of 
a scheme which is based on a combination of the two postulates 
of altered neuron properties and activity of closed chains. 
From this he is able to derive the effect of stimulus strength, 
the time (or trials) necessary for conditioning, and the in- 
fluence of the interval between trials. 

If learning (according to the closed chain theory) is as- 
sumed to depend on continuing activity in closed chains when 
once excited, it might be supposed that anesthetics, interfer- 
ing with states of excitation, should produce forgetting. A 
possible objection to the theory may be based on the fact that 
anesthetics do not produce forgetting. It is known, for ex 
ample, that cortical action potentials are greatly modified un- 
der deep anesthesia (Derbyshire, Rempel, Forbes, and Lam- 
bert, 1936), yet anesthetics do not result in loss of previous 
learning. 

Influence of Bioelectric Fields. The theories outlined 
above all assume that the modification in threshold of a 
neuron is a function of impulses traversing the synapse- Any 
alternative would involve some sort of “action at a distance.” 


NEUROPHYSIOLOGICAL MECHANISM 331 


Neural activity is accompanied by changes in electrical po- 
tential, and activity in large numbers of contiguous neurons 
will establish relatively great potential differences which might 
alter the threshold of unactivated neurons lying within the 
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SCHEMATIC ARRANGEMENT OF NEURONS To Account FOR CONDITIONING BY 
CLosep CHAINS 


The assumption is made that activation of a nerve cell occurs only when 
the cell receives excitation from two axon endings simultaneously. The 
symbols are the same as in Figure 38. Impulses in CS are ineffective with 
respect to neurons a and b. If impulses in UncS and CS arrive simultaneously, 
however, they summate to excite b. This sets the closed chain b-c in con- 
tinuous activity, and impulses in CS now summate with collaterals from 
fibers in the chain to activate a and Runes. Simultaneous excitation of b 
will occur by chance if CS and UncS are stimulated together repeatedly. 
The chances of simultaneity are increased if the frequency of impulses in 
CS and UncS is greater; i.e., if the intensity of the stimuli is greater. While 
the speed of conditioning in any single neuron unit is thus largely a matter 
of chance according to this scheme, the sum of the changes in many such 
units would result in a gradual increase in the number of Rones fibers acti- 
vated. 

This scheme is not elaborated here to account for extinction, or other 
phenomena of conditioning. Inhibitory effects might be introduced by the 
addition of specific inhibitory collaterals, or by consideration of temporal 
relations resulting in refractory period decrement. 


electrical field. Such a conception is implied in certain Gestalt 
theories (Köhler, 1929; Lashley, 1930a; Gengerelli, 1934; 
Washburne, 1935), but its application has never been worked 
out in detail. 

The above theories all attempt to account for classical con- 
ditioning, and nothing except pure substitution is encompassed 
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by any of them. Neurological theories of the mechanism of 
the principle of effect may first reduce effect to substitution 
(Holt, 1931), or may attempt to formulate the action of ef- 
fect directly in neural terms. Troland’s (1928) retroflex theory 
is the most complete presentation of the latter position. Learn- 
ing of the trial-and-error variety is analyzed first into the in- 
crease or decrease in conductance of the neural pathways in- 
volved in specific responses. Troland speaks of punishments 
as nociceptive stimuli, and rewards as beneceptive stimuli. 
His neural theory is that “the stimulation of nociceptive af- 
ferent channels reduces the conductances of the paths of 
cortical conduction which are operative at the moment, or 
have been operative over a limited prior interval; and that, 
on the other hand, the excitation of beneceptive channels will 
cause a corresponding increase in conductances” (1928, p. 
205). He recognizes that the changes in conductance must be 
relatively permanent and must be cumulative over succes- 
sive trials, and suggests several possible mechanisms for such 
changes. There might be inhibitory and facilitatory nerve 
fibers from the affective centers of the thalamus to all the as- 
sociation points of the cortex, or there might be a general dif- 
fuse discharge from the thalamus which would be inhibitory 
at a high frequency and facilitating at a low frequency. Or the 
thalamus might act to control the liberation of two different 
kinds of hormones which would circulate in the blood stream 
and facilitate or inhibit concurrent cortical activity. These sug- 
gestions are not supported by any direct evidence at the pres- 
ent time. One objection to the general scheme is that it renders 
impossible the simultaneous strengthening of one response 
and weakening of another. 

Actual instances of learning are of course much more com- 
plicated than the discussion of neural theories would imply- 
Any stimulus-response unit involves a large number of neu- 
tons on both the afferent and efferent sides. For the sake of 
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simplicity, the mechanism of threshold change has been dis- 
cussed in a single neuron unit. From the all-or-none law of 
excitation, it will be seen that activation by the conditioned 
stimulus will occur suddenly in any single neuron, although a 
number of repetitions of the stimuli may be required before 
the critical point is reached. The conditioned response will ap- 
pear as overt activity only when a sufficient number of the 
neuron units are activated by the conditioned stimulus. It 
is a reasonable assumption that these various units will re- 
quire different amounts of training (Hoagland, 1930), the 
result of which would be a gradual curve of acquisition such 
as is ordinarily obtained. The activity of single motor units in 
the conditioned flexion response of the rat has been recorded 
by Hunter (1937) from concentric needle electrodes placed 
in the gastrocnemius muscle. Conditioning was apparent as 
an increase in the number of active units, and an increase in 
frequency and decrease in latency of any single unit. Extinc- 
tion presented the opposite picture: a decrease in the number 
of active units, and a decrease in frequency and increase in 
latency of any single unit. 

Opposition has frequently been raised to any explanation 
of learning in terms of specific synapses whose traversibility 
is altered by the training procedure. It has been pointed out 
that a learned response may be evoked by stimulation of af- 
ferent conduction paths which were not active during the 
process of learning. Moreover, the adequate stimulus for a 
learned response is not necessarily one which stimulates cer- 
tain particular afferent neurons. The facts of relative dis- 
crimination, transposition, perceptual constancy, and so forth, 
indicate that the stimulus can be better designated as a ratio 
of intensities distributed in time or space. These considera- 
tions place definite limitations upon the form of a neural 
theory, but do not preclude the possibility that modification 
takes place in specific pathways. Equivalent stimuli which 
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activate different absolute groups of afferent neurons presum- 
ably have some central conduction pathway in common, and 
the learning modification would occur in that part of the 
pathway which is common to both stimuli. No alternative to 
the theory of modification of synaptic resistance has ever 
been stated in a form specific enough to permit critical evalua- 
tion. 

Inhibition. It should be made clear again that the validity 
of the concept of inhibition as applied to behavioral observa- 
tions in no way rests on any particular theory of the neural 
mechanism of inhibition. In the past fifty years new physio- 
logical explanations of inhibition have replaced the old with 
regularity. None of these changes has affected the psychologi- 
cal concept in other than an analogical manner. Current neural 
theories are perhaps in greater flux than ever before, and it 
is beyond the province of this discussion to attempt to evalu- 
ate them. 

In the discussion of the nature of extinction the statement 
was made that repetition of any response leads to adaptation 
unless some other influence such as reinforcement counteracts 
the effect. This has been repeatedly observed in studies of re- 
flex conduction in the spinal cord, and more recently has been 
carefully studied in the cortex (Dusser de Barenne and W. S. 
McCulloch, 1937, 1939). After repetitive electrical stimulation 
of a point on the motor cortex of a monkey, there is a period 
of reduced excitability which is most pronounced after several 
seconds, and which may be distinguished from reciprocal 
inhibition, fatigue, and refractory phase. The degree of re- 
duced excitability is a function of the voltage, wave-shape, 
frequency, and duration of the electrical stimulation. The 
effect is never absolute and may be overcome by an increase 


in strength of the second stimulus or by the summation of 
two stimuli. 
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Summary 


Pavlov’s theory of cortical activity is expressed in terms 
of excitation and inhibition which interact through irradiation, 
concentration, and induction. There are definite limitations 
to the validity of Pavlov’s conceptions of neural function in 
conditioning. 

The earlier insistence that the cortex was essential for con- 
ditioning has now given way before satisfactory evidence that 
conditioning may be mediated by subcortical structures. This 
has in turn made unnecessary the postulation of vicarious 
functioning to account for learning in the absence of given 
cortical areas. The cortex remains important in normal con- 
ditioning, and, according to the principle of functional en- 
cephalization, its importance increases throughout the phylo- 
genetic series to man. 

Experimental demarcation of the anatomical locus of con- 
ditioning suggests that the essential modification is central. 
Peripheral structures can be largely dispensed with in the 
formation of conditioned responses. Moreover, the association 
appears to lie outside the regular pathways of the conditioned 
stimulus, the unconditioned stimulus, and the unconditioned 
response. The remaining possibility is an association pathway, 
but only fragmentary direct evidence for it now exists. This 
does not imply a general association area. 

Speculations regarding the nature of synaptic modification 
include anatomical growth of axons and dendrites, changes in 
the physico-chemical properties of the nerve cell, continued 
activity in closed neural chains, and the influence of bioelectric 
fields. Evidence is lacking for any of the proposed explana- 
tions, and at best the speculations serve only to account for 
stimulus-substitution which has been shown to be but one of 
several forms which conditioning takes. 
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Eventual explanation of conditioning in neurological terms 
depends upon the convergence of two experimental trends. 
In the first place, the psychologist must furnish a more pre- 
cise statement of the fundamental factors in response modifi- 
cation, defining, for example, the effect of reinforcement for 
the simplest cases. With the behavioral facts in better order, 
the task of explanation faced by the neurophysiologist will be 
clearer. In the second place, neurophysiological study of long- 
lasting modifications in synaptic function will accumulate the 
necessary data for a verifiable explanation of these simplest 
instances of learning. 


Notes 


In the present status of knowl- 
edge, neural theory is not basic to 
conditioning theory. The known 
facts of neural function cannot be 
utilized to predict or to limit the re- 
sults of behavioral studies. None of 
the several principles and facts pre- 
sented in earlier chapters can be de- 
duced from present direct knowledge 
of the nervous system. Even the bas- 
ic law that a response varies in mag- 
nitude with intensity of stimulus 
would be equally true if the nerves 
were copper wires or pneumatic 
tubes. The facts of speed of conduc- 
tion and synaptic delay cannot pre- 
dict the latency of a conditioned re- 
sponse, for we have no idea what 
length of nerve or how many synap- 
ses are involved. New findings may 
reduce the temporal summation in- 
terval of a synapse from 40 msec. to 
0.2 msec. without altering the prin- 
ciple of temporal summation of stim- 
uli. The demonstration of adapta- 
tion of synapses yields no clue to 
the prediction that the eyelid reflex 


will adapt more rapidly to light than 
to an air-puff. Many of the so-called 
neural facts, such as reflex inhibi- 
tion, which seem most relevant to 
conditioning are in reality behavior- 
al laws stated as relations between 
afferent and efferent nerve activity 
without direct observation of any 
intermediate neural events (Skin- 
ner, 1931). This point of view of 
course does not preclude the possi- 
bility that on the basis of future 
work neurological prediction of be- 
havioral facts may be achieved. 


Pavloy’s Theory of Cortical 
Functioning 


Pavlov’s theory of cortical func- 
tioning is presented in his two books 
(1927, 1928). Critical discussions 
and supplementations of his con- 
ceptions are presented by ee 
(1924, 1927), Borovski (1929), 
Razran (1930), | Denny-Brown 
(1932), and Bekhterev (1932)- 
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Anatomical Localization of 
Conditioning 


The utilization of training proce- 
dures in the study of the localization 
of cerebral functions was undertaken 
at approximately the same time by 
Pavlov, Bekhterev, and Franz. Ref- 
erence will be made here only to the 
work on mammals. Learned respons- 
es to auditory stimuli after removal 
of the temporal lobes have been 
demonstrated in dogs by Krijanov- 
sky and Babkin in Pavlov’s labora- 
tory (Pavlov, 1927, p. 332-334), 
Kalischer (1909), Swift (1912), al- 
though Rothmann (1908) and Suth- 
erland and Dworkin (1932) report- 
ed negative results; in rats by Wiley 
(1932), Pennington (1937). Condi- 
tioned responses to olfactory stimu- 
li have been demonstrated in dogs 
after removal of the pyriform lobes 
by Zavadsky (1910), after removal 
of the anterior half of the cortex by 
Saturnov and by Kouraev (Pavlov, 
1927, p. 371), after transection of 
both fornices by W. F. Allen (1938), 
and in rats after destruction of large 
portions of the cerebrum by Swann 
(1935). Successful training in optic 
responses after removal of the area 
striata has been reported in rats by 
many researchers following work by 
Lashley (1929), in cats by Ken- 
nedy (1939), Ten Cate (19342, 
1939), and K. U. Smith (1937), in 
dogs by Koudrin (Pavlov, 1927, pp. 
331, 343), Kalischer (1909), D. G. 
Marquis (1934), D. G. Marquis 
and Hilgard (1936), and in mon- 
keys by Rosenzweig (1935), Klüver 

(1936), and D. G. Marquis and 
Hilgard (1937). Proprioceptive and 
cutaneous conditioned responses aft- 
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er removal of portions of the sensori- 
motor cortex have been studied in 
rats by D. E. Smith (1939), in 
cats by Storey (1937), in dogs by 
several of Pavlov’s students (Pav- 
lov, 1927), and in primates by T. C. 
Ruch and Fulton (1935), T. C. 
Ruch, Fulton, and German (1938), 
and Harlow and Settlage (1936). 

The effect of destruction of the 
cortical motor center for the uncon- 
ditioned response has been studied 
by several investigators. Pavlov 
(1928, p. 97-98) reported that con- 
ditioned salivation was unaffected 
by bilateral removal of the cortical 
salivary center. Bilateral removal 
of the motor center for blinking 
(pre-cruciate cortex) in dogs did not 
affect the formation of conditioned 
eyelid responses (D. G. Marquis 
and Hilgard, 1936). Surgical extir- 
pation of the motor cortex in 
monkeys has been shown by Lash- 
ley (1924) and Jacobsen (1934) to 
result in no loss of learned motor 
performances involving the hands. 
Asratian (1935), Ten Cate (1938) 
and Rosenthal (1938) found that 
conditioned leg withdrawal was not 
abolished by removal of the motor 
cortex bilaterally in dogs, but Bekh- 
terev (1923b) and W. F. Allen 
(1938) reported a severe impair- 
ment, corresponding to that ob- 
served after complete decortication 
by Culler and Mettler (1934). This 
problem needs further research. The 
effect of removal of one entire hemi- 
sphere in dogs has been studied by 
Blagoveshchenskaya (1929), Ro- 
senthal (1937), Rosenzweig (1935), 
Poltyrev and Alexejev (1936). 

The possibility of establishing 
conditioned responses in the isolated 
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spinal cord has been demonstrated 
recently by the experiments of Cul- 
ler (1937), and Shurrager and Cul- 
ler (1938) on flexion responses in 
spinal dogs. A pressure stimulus at 
the tip of the tail was followed re- 
peatedly by a shock to the paw of 
one hind leg. No conditioned re- 
sponse involving movement of the 
limb developed, but when the semi- 
tendinosus muscle was exposed and 
directly observed, a small twitch of 
one end of the muscle was apparent. 
Extinction and spontaneous recov- 
ery of this response could be ob- 
tained. Further experimental study 
is necessary before the relation of 
these results to conditioning in gen- 
eral becomes clear. 


Functional Components of the 
Conditioning Mechanism 


The only thorough and compre- 
hensive treatment of this phase of 
the subject is by Beritov (1924, 
1927). No attempt has been made 
here to repeat his theory in detail, 
but rather to summarize the experi- 
mental studies which have appeared 
subsequently. 

Non-neural conditioning, in which 
the agent producing the uncondi- 
tioned response acts through the 
blood stream without intervention 
of the central nervous system, is im- 
plied in studies of conditioned vege- 
tative changes such as leucocytosis, 
antibody formation, agglutination, 
diuresis, and pancreatic secretion. 
These studies have been summarized 
by Hull (1934c), Loucks (19372), 
Metalnikov (1934), Razran (1933a) > 
and Razran (1937) gives a full list 
of references. Their interpretation 
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is not clear at present. In several 
instances the results have been ques- 
tioned because of inadequate experi- 
mental controls, and in other in- 
stances the possibility has not been 
excluded that the agent producing 
the unconditioned response actually 
stimulates neural centers to produce 
the reaction reflexly. 

Although the essential modifica- 
tion in conditioning is central, it is 
still possible that such changes are 
reflected in peripheral function. Dra- 
bovitch and coworkers have dem- 
onstrated changes in the chronaxie 
of the muscles of the limb coinci- 
dent with the elicitation of a de- 
layed conditioned leg withdrawal 
response in dogs. The series of exper- 
iments published during the past 
four years are summarized in a re- 
cent article (Drabovitch, 1937). 
During the interval following the 
onset of the conditioned stimulus (a 
prolonged tone), the chronaxie of 
both flexors and extensors of the leg 
increases to a value which may be 
double the resting value. During the 
conditioned reaction, the chronaxie 
decreases. These changes are paral- 
leled in the chronaxie of the cortical 
motor point for flexion of the limb, 
so that an approximate isochronism 
of the motor pathway exists. The 
interpretation of the chronaxie meas- 
ure (H. Davis and Forbes, 1936) 18 
not clear enough to justify further 
speculation concerning a mechanism 
of conditioning. 


The Nature of Synaptic Modi- 


fication 


Reviews of the recent advances aa 
the knowledge of synaptic transm 
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sion may be found in Fulton (1938), 
Lorente de Nó (1938a, 1938b, 
1938c), and Gasser (1937). À sym- 
posium on the synapse by a num- 
ber of leading neurophysiologists ap- 
peared in the September, 1939, issue 
of the Journal of Neurophysiology. 
(Vol. 2, No. 5.) References to the 
earlier, speculative theories of the 
synaptic modification in learning 
may be obtained from Lashley’s re- 
view (1934). 

Mathematical derivation of the 
possibilities of neuron circuits has 
been presented in full detail in a re- 
cent book by Rashevsky (1938). 
Extensions and elaborations have 
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subsequently been published by 
Householder (1938a, 1938b) and 


Landahl (1938a, 1938b). 

The requirements and present al- 
ternatives for a theory of central in- 
hibition have been reviewed by Ful- 
ton (1938). The rapidly growing 
evidence that the properties of the 
synapse do not differ qualitatively 
from those of peripheral nerve lends 
special weight to theories expressed 
in terms of the positive after-poten- 
tial of internuncial neurons (Gas- 
ser, 1937), or of the temporal rela- 
tions of impulses summating at a 
synapse (Lorente de Nó, 1938c). 





Glossary 


Prominent in the glossary are terms specific to conditioning, including 
translations of Pavlov’s terms, which are designated by his name. The index 
may serve in locating the usage of words and expressions not included in 


the glossary. 


adaptation. The decrement in a 
response which is a consequence of 
its repeated elicitation. (Syn. in- 
trinsic inhibition; cf. internal 
inhibition; contr. w. extrinsic 
inhibition, interference) 


association. The establishment 
of functional relations between psy- 
chological activities and states in the 
course of individual experience. A 
broader term which includes con- 
ditioning. (Cf. conditioning) 


associative shifting (Thorn- 
dike). Considered by Thorndike to 
be the more general case of which 
the conditioned response is a special 
case. If a situation S; originally 
evokes Rx, ie., S: -> Rx, then Sı 
may be made to evoke Rx by addi- 
tions until $:+S:—> Rx and later 
subtractions until Sa -> Rx. (Contr. 
w. trial-and-error learning) 


automatic act. An involuntary 
act occasioned by the stimulus, dis- 
tinguished from a reflex in that it 
has been acquired in the course of 
individual experience. (Cf. reflex) 


avoidance training. A training 
procedure in which the learned 
movement circumvents or prevents 
the appearance of a noxious stimu- 
lus. (Distg. fr. instrumental es- 
cape training) 


backward conditioning. The 
experimental arrangement in which 
the conditioned stimulus is presented 
after the cessation of the uncondi- 
tioned stimulus. 


choice-reaction discrimina- 
tion, see discrimination 


classical conditioning exper- 
iment. An experiment after the pro- 
totype of Pavlov, which consists in 
the repeated presentation of the 
conditioned and unconditioned stim- 
uli in a controlled relationship so 
that there occur alterations in reac- 
tion tendencies with respect to the 
conditioned stimulus which would 
not arise except for its relationship 
to the unconditioned stimulus and 
response. Distinguished from instru- 
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mental reward and escape training 
and from avoidance training in that 
the conditioned response neither de- 
livers nor prevents the appearance 
of the unconditioned stimulus. (Cf. 
avoidance training; instrumen- 
tal reward training; instru- 
mental escape training; sec- 
ondary reward training) 


complete substitution, see 
redintegration 


concentration (Pavlov).1. The 
focussing of the hypothetical nervous 
processes of excitation and inhibi- 
tion within restricted cortical areas. 
(Contr. w. irradiation) 2. Behav- 
iorally, restricted generalization. 
(Cf. generalization) 


conditioned inhibition (Pav- 
lov). A variety of internal inhibition 
defined by the experimental arrange- 
Ments under which it is obtained. 
The negative, non-reinforced, com- 
bination to be differentiated from 
the positive conditioned stimulus 
consists of the positive stimulus plus 
an added stimulus known as the 
conditioned inhibitor. 


conditioned response. A re- 
sponse which appears or is modified 
as a consequence of the occurrence 
of a conditioned stimulus in prox- 
imity to reinforcement. (Syn. con- 
ditioned reflex or conditional 
reflex [Pavlov]; association-re- 
flex [Bekhterev]; individual re- 
flex [Beritov]) 


conditioned stimulus (Pav- 
lov). The experimental stimulus to 
which a new or modified response 
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becomes related through the process 
of conditioning. (Syn. conditional 
stimulus, inadequate stimulus, 
substitute stimulus) 


conditioning. The process of 
training which results in the forma- 
tion of conditioned responses. (Cf. 
association) 


counter conditioning. Extinc- 
tion under circumstances in which 
the response decrement is hastened 
by the reinforcement of a response 
which displaces the original condi- 
tioned response. (Distg. fr. adapta- 
tion; cf. interference) 


delayed conditioning (Pav- 
lov). The conditioned stimulus be- 
gins from 5 seconds to several 
minutes before the unconditioned, 
and continues until reinforcement 
occurs. (Distg. fr. trace condi- 
tioning; not to be confused with 
delayed reaction experiment 
[ Hunter] ) 


derived reinforcement. The 
strengthening of a conditioned Te- 
sponse by reinforcement with a stim- 
ulus which depends for its reinforc- 
ing value on prior association with a 
reward or punishment. (Contr. W- 
homogeneous reinforcement; 
heterogeneous reinforcement; 
cf. avoidance training, second- 
ary reward training) 


differential inhibition (Pav- 
lov). A form of internal inhibition 
arising in respect to the negative 
stimulus when stimuli are discrimi 
nated by the method of contrasts. 
(Cf. discrimination) 
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discrimination. The inferred 
basis for differential responses to un- 
like stimuli. Discrimination is al- 
ways measured by differential re- 
sponses, but absence of a specified 
differential response is not necessar- 
ily an indication of lack of discrim- 
inatory capacity. The method of 
contrasts refers to an arrangement 
for studying discrimination in which 
one of a pair of stimuli (the posi- 
tive stimulus) is invariably rein- 
forced, the other (the negative 
stimulus) invariably non-reinforced. 
Discrimination is inferred from the 
greater strength of conditioning to 
the positive than to the negative 
stimulus following this procedure. 
Choice-reaction discrimina- 
tion refers to the method in which 
a particular response is demanded to 
one of a pair of stimuli, a different 
response to the other member of the 
pair, correct responses to either stim- 
ulus being rewarded. 


disinhibition (Pavlov). A tem- 
porary increase in the strength of 
an extinguished conditioned response 
as the result of an extraneous stimu- 
lus. While called by Pavlov an “in- 
hibition of inhibition,” the term 
may be used in a strictly behavioral 
sense. (Contr. w. external inhibi- 
tion) 


dominance (Ukhtomsky). A 
physiological description of the in- 
teraction of certain reflexes, whereby 
following repeated stimulation, the 
dominant one is evoked by stimuli 
appropriate to other reflexes, while 
at the same time the normal re- 
sponses to these stimuli are inhib- 
ited. (Sometimes used incorrectly 
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as a synonym for prepotency [Sher- 
rington ]) 


effect principle. The principle 
that the tendency to repeat an act 
depends on its proximity to a rein- 
forcing state of affairs. While the 
principle is particularly appropriate 
in the explanation of instrumental 
reward and escape learning, in 
which reinforcement is a conse- 
quence of the conditioned response, 
the principle does not demand that 
the learned act be instrumental to 
reinforcement. (Cf. goal-gradient 


[Hull]; distg. fr. expectancy 
principle; substitution prin- 
ciple) 


excitation (Pavlov). One of two 
hypothetical nervous processes in 
the cortex, interacting with inhibi- 
tion, its opposite. (Cf. inhibition) 


expectancy principle. The 
principle which interprets the ani- 
mal’s learned behavior as consonant 
with anticipated (expected) events 
or consequences. (Syn. sign learn- 
ing [Tolman]; distg. fr. effect 
principle, substitution prin- 
ciple) 


experimental extinction, see 
extinction 


experimental neurosis (Pav- 
lov). A disturbed state appearing in 
some animals as a result of condi- 
tioning procedures, such as too dif- 
ficult discriminations, or prolonged 
periods of delay. The animal is un- 
suitable for experimentation, but 
may or may not appear to be dis- 
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turbed when outside the experi- 
mental situation. 


external inhibition (Pavlov). 
À temporary reduction in the 
strength of a conditioned response 
as a result of the occurrence of an 
extraneous stimulus. (Contr. w. dis- 
inhibition) 

extinction (Pavlov). The spe- 
cific procedure of presenting the 
conditioned stimulus unaccom- 
panied by the usual reinforcement; 
also the decrement in a conditioned 


response which results from that 
procedure. 


extinctive inhibition (Pav- 
lov). The variety of internal inhibi- 
tion which appears with non- 
reinforcement of a conditioned re- 
sponse. (Cf. extinction) 


extrinsic inhibition, see inter- 
ference 


facilitation. A temporary non- 
learned increase in strength of re- 
sponse as a result of the occurrence 
of a second stimulus or response. 
(Distg. fr. reinforcement) 


forgetting. Decrement in a re- 
sponse when elicited after an inter- 
val during which it has not been 
elicited. Forgetting differs from ex- 
tinction and other forms of adapta- 
tion through repetition. (Cf. adap- 
tation; extinction; retroactive 
inhibition) 


fractional component condi- 


tioned response. That result 
within classical conditioning in 
which the conditioned response, 
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while not fully duplicating the un- 
conditioned response, is best de- 
scribed as resembling it or some of 
its components. (Syn. fractional 


anticipatory goal response 
[Hull]; cf. preparatory re- 
sponse; redintegrative re- 
sponse) 


generalization (Pavlov). The 
behavioral fact that a conditioned 
response formed to one stimulus 
may also be elicited by other stim- 
uli which have not been used in the 
course of conditioning. If one stim- 
ulus is like another except for a 
difference in a single dimension (e.g., 
pitch or loudness of tones as de- 
termined by psychophysical experi- 
ments) the degree of generalization 
will vary inversely with the distance 
of the stimuli from each other along 
this dimension. The function ex- 
pressing this relationship is called 
the gradient of generalization 
of conditioning. Its form varies 
with circumstances of experimenta- 
tion. There is a corresponding gra- 
dient of generalization of ex- 
tinction resulting when each of 
several similar stimuli evokes a con- 
ditioned response, and the response 
to one of them is extinguished. 
Stimuli otherwise unlike may par- 
ticipate in generalization through 
the equivalence of the responses 
which they evoke. Such generaliza- 
tion is known as mediated gener- 
alization. (Cf. irradiation; i 
sponse generalization; spaāt!a 
generalization) 


goal gradient. (1) An ene 
and somewhat specialized statemen 
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of the gradient of reward (Hull). 
(2) A performance gradient in which 
parts of a serial act are reinforced 
by a common goal situation, acts 
being more vigorous the nearer they 
lie to the reinforcing state of affairs. 
The goal-gradient so defined is ex- 
plained in part by the gradient of 
reward, but is a gradient of strength 
of response rather than of strength 
of conditioning and is modified by 
factors other than proximity of in- 
dividual responses to the reinforce- 
ment. (Cf. gradients of rein- 
forcements; spread of effect) 


gradient of generalization, 
see generalization 


gradients of reinforcement. 
The class of functions describing the 
effect of proximity of reinforcement 
on strength of conditioning. Among 
these is the temporal gradient 
within classical conditioning, 
referring to the influence of interval 
between conditioned and uncondi- 
tioned stimuli. A short forward or- 
der of a few tenths of a second is 
usually found most advantageous. 
Also included is the temporal gra- 
dient within avoidance learn- 
ing, which refers to the influence 
of the interval between the condi- 
tioned stimulus and the punishment. 
The gradient of reward has both 
temporal and non-temporal aspects. 
A conditioned response is strength- 
ened by the reward type of reinforce- 
Ment in inverse proportion to the 
time elapsing between the response 
and the reinforcement, strengthen- 
ing being less as the reward is de- 
layed. This is the temporal aspect of 
the gradient of reward. A similar 
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gradient is found for non-temporal 
aspects of the separation of response 
and reward, when proximity is de- 
fined in terms of distance or in terms 
of number of responses removed. The 
temporal gradient in delayed 
reaction experiments is not 
strictly a gradient of reinforcement, 
since the delay is introduced be- 
tween the presentation of a discrim- 
inatory conditioned stimulus and the 
opportunity to respond to it. (Cf. 
goal-gradient; spread of ef- 
fect) 


habit-family hierarchy (Hull). 
A number of habitual alternative 
behavior sequences having in com- 
mon the initial stimulus situation 
and the final reinforcing state of af- 
fairs, the alternative sequences hav- 
ing a preferential order which con- 
stitutes them a hierarchy. 


heterogeneous reinforce- 
ment. The strengthening of a con- 
ditioned response through reinforce- 
ment which depends on a response 
which does not resemble the condi- 
tioned response. This is characteris- 
tic of reward and escape learning, 
but not limited to it. (Contr. w. 
homogeneous reinforcement; 
derived reinforcement) 


higher-order conditioned re- 
sponse (Pavlov). A conditioned 
response based on reinforcement by 
another conditioned stimulus and 
response. A first-order response is 
reinforced by an unconditioned stim- 
ulus, a second-order response (or 
secondary response) by a first-order 
conditioned stimulus, and so on. 
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homogeneous reinforcement. 
The strengthening of a conditioned 
response by reinforcing with an un- 
conditioned stimulus which evokes 
activity similar to the conditioned 
response. Required according to the 
substitution principle. (Contr. w. 
heterogeneous reinforcement; 
derived reinforcement) 


inhibition. 1. (Pavlov). One of 
the two hypothetical cortical proc- 
esses, interacting with excitation, its 
opposite. 2. Behaviorally, any de- 
crease in the strength of a response 
which is occasioned by positive stim- 
ulation, either of the response itself 
(as in extinction) or of some other 
response (as in external inhibition). 
(Cf. conditioned inhibition, dif- 
ferential inhibition, external 
inhibition, extinctive inhibi- 
tion, extrinsic inhibition, inhi- 
bition of delay, inhibition of 
reinforcement, internal inhibi- 
tion, intrinsic inhibition) 


inhibition of delay (Pavlov). 
That form of internal inhibition 
manifested during the latent period 
of a delayed conditioned response. 
That the interval of delay is inhibi- 
tory is evidenced by (1) the ease 
with which disinhibition may be 
demonstrated, and (2) secondary 
inhibition of weak conditioned re- 
sponses if the attempt is made to 
elicit them during the period of 
delay. 


inhibition of reinforcement 
(Hovland). The diminution of a 
conditioned response which may oc- 
cur within reinforcement trials if 
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such trials are repeated at short 
intervals. The phenomenon is like 
adaptation, in that it is temporary, 
and may be recovered from by rest. 


instrumental escape train- 
ing. That form of training in which 
the strengthened act terminates a 
noxious stimulus to which the or- 
ganism has been reacting. (Distg. 
fr. avoidance training) 


instrumental reward train- 
ing. That form of learning in which 
the strengthened act (usually ma- 
nipulation or locomotion) produces 
an unconditioned stimulus capable 
of satisfying a drive. (Cf. avoid- 
ance training; instrumental 
escape training; secondary re- 
ward training; trial-and-error 
learning) 


interference. The decrement in 
a response which is a consequence 
of the elicitation of another incom- 
patible response. (Syn. extrinsic 
inhibition; cf. adaptation, ex- 
tinction) 


internal inhibition (Pavlov). 
That form of inhibition which de- 
pends upon stimulation by a con- 
ditioned stimulus. (Contr. w. €X- 
ternal inhibition) 


intrinsic inhibition, see adap- 
tation 


involuntary response. A Te- 
sponse which, regardless of its earlier 
history, is now largely determined 
by the experimental stimulus and 1s 
relatively slightly affected by 1 
structions or substitutes for instruc 


GLOSSARY 


tions. (Cf. voluntary response; 
semi-voluntary response) 


irradiation (Pavlov). 1. The 
wave-like spread of excitation and 
inhibition through the cortex, the 
inferred physiological basis for gen- 
eralization. (Contr. w. concentra- 
tion) 2. Behaviorally, used chiefly 
in reference to spatial generalization. 
(Cf. generalization; spatial 
generalization) 


learning. Change in the strength 
of an act through training proce- 
dures (whether in the laboratory or 
the natural environment) as dis- 
tinguished from changes in the 
strength of the act by factors not 
attributable to training. 


mediated generalization, see 
generalization 


mosaic of functions (Pavlov). 
A conception of localization of func- 
tions in the cortex. For Pavlov, the 
mosaic is determined in part an- 
atomically (the analyzers), in part 
functionally, as a result of the in- 
tegration of conditioned excitation 
and inhibition. 


negative adaptation, see 


adaptation 


negative conditioned stimu- 
lus. A conditioned stimulus which, 
through repeated presentation, either 
alone or in combination, without 
being followed by reinforcement, 
tends to be followed by reduced or 
inhibited conditioned responses. 
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negative induction (Pavlov). 
The intensification of inhibition un- 
der the influence of preceding exci- 
tation. (Contr. w. positive induc- 
tion) 


operant (Skinner). An act which 
first appears as a random move- 
ment. The operant commonly be- 
comes related to a discriminated 
stimulus, which then sets the occa- 
sion for the operant behavior, but, 
strictly speaking, does not elicit it. 
(Cf. random movement; contr. w. 
respondent) 


paradoxical phase (Pavlov). A 
state of the cortical processes lying 
between that of equalization and 
complete inhibition in which only 
weak conditioned stimuli lead to re- 
sponses and strong stimuli either 
elicit no responses at all or have a 
barely noticeable effect. (Cf. phase 
of equalization; ultra-paradox- 
ical phase) 


partial reinforcement. A modi- 
fication of simple conditioning in 
which the reinforcement occurs on 
only a fraction of the trials. Under 
some circumstances random alterna- 
tion of reinforcement and non-rein- 
forcement leads to a normal acquisi- 
tion rate, and at the same time 
increases resistance to extinction. 
Two special forms of partial rein- 
forcement within instrumental train- 
ing lead to unlike consequences. Pe- 
riodic reconditioning (Skinner) 
refers to the reinforcing of instru- 
mental acts according to a time 
schedule, such as reinforcement ev- 
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ery 3 minutes. Under such circum- 
stances, the longer the interval be- 
tween reinforcements, the less the 
rate of responding. Reinforcement 
at a fixed ratio (Skinner) refers 
to the reinforcing of instrumental 
acts according to a schedule de- 
termined by the number of responses, 
such as reinforcement every fifth re- 
sponse. Under these circumstances, 
the higher the ratio (i.e., the fewer 
the reinforcements) the greater the 
rate of responding. 


periodic reconditioning, see 
partial reinforcement 


phase of equalization (Pav- 
lov). A state of the cortical processes 
lying between that of normal wak- 
ing and sleeping in which all condi- 
tioned stimuli, independent of their 
intensities, are equally effective. (Cf. 
paradoxical phase; ultra-par- 
adoxical phase) 


positive induction (Pavlov). 
The intensification of excitation un- 
der the influence of preceding inhi- 


bition. (Contr. w. negative in- 
duction) 


preparatory response. A form 
of conditioned response within clas- 
sical conditioning, often unlike the 
unconditioned response, which may 
be appropriately characterized as a 
Tesponse in readiness for the appear- 
ance of the unconditioned stimulus 
(Cf. fractional component re- 


Sponse; redintegrative re- 
sponse) 


Primary punishment. A stim- 
ulus capable of reinforcing avoidance 
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behavior because of defensive re- 
sponses which it evokes, rather than 
because of prior association with 
other noxious stimuli. (Cf. primary 
reward) 


primary reward. A stimulus 
deriving its reinforcing value from 
the fact that it satisfies an organic 
need, rather than from the fact of 
prior association with other need- 
satisfying stimuli. (Cf. secondary 
reward) 


pseudo-conditioning (Gre- 
ther). The strengthening of a re- 
sponse to a previously neutral stim- 
ulus through the repeated elicitation 
of the response by another stimulus, 
without paired presentation of the 
two stimuli. (Cf. dominance; sen- 
sitization) 


pseudo-reflex (Skinner). A re- 
lation between a stimulus and a re- 
sponse which superficially resembles 
a reflex by Skinner’s definition, but 
depends upon or involves other terms 
than those expressed in the relation. 
The clearest example is the discrim- 
inated operant. (Cf. operant) 


pure-stimulus act (Hull). An 
act which serves no biological func- 
tion other than to provide stimuli 
which aid in the integration of the 
organism’s behavior. 


random movement. An act for 
which no stimulus determinant can 
be specified. The random charactet 
disappears as soon as the act be- 
comes correlated with an observ- 
able stimulus. (Syn. spontaneous 
movement; cf. operant) 
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recency principle (Guthrie). A 
principle explanatory of associative 
learning which states that an organ- 
ism when next confronted with a 
stimulating situation closely resem- 
bling an earlier one tends to react 
as it last did in such a situation. 

To be distinguished from that 
form of the ‘law of recency’ which 
states that associative strength is a 
function of the time elapsed since 
the association was established. 
Guthrie’s principle is that of recency 
in order of occurrence, not of re- 
cency in time. 


reconditioning. The reestab- 
lishment of a conditioned response 
by reinforcement after it has been 
diminished by extinction or forget- 
ting. 


redintegration. The arousal of 
a response by a fraction of the stim- 
uli whose combination originally 
aroused it. (Cf. substitution prin- 
ciple) 


redintegrative response. That 
form of conditioned response in clas- 
sical conditioning which is essentially 
a replica of the unconditioned re- 
sponse. (Cf. fractional compo- 
nent response; preparatory re- 
sponse) 


reflex. 1. Preferably, a stimulus- 
determined act which can occur 
without specific training of the or- 
ganism. 2. Any of the motor activi- 
ties of the body, whether simple or 
complex, to the extent that they 
may be considered as determined 
(Pavlov). Hence, exploratory re- 
flex, freedom reflex, social 
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reflex, etc. 3. The analytical unit 
of behavior for describing the ob- 
served relation between a stimulus 
and a response, when the correla- 
tion between stimulus and response 
is lawful (Skinner). 


reflex reserve (Skinner). The 
total number of times a conditioned 
response will occur without further 
reinforcement, granted that the en- 
vironmental conditions are adequate 
for its repetition. Hence, one meas- 
ure from which to infer strength of 
conditioning. (Cf. strength of 
conditioning) 


reinforcement (Pavlov). 1. 
The experimental arrangement of 
presenting the conditioned stimulus 
accompanied by the unconditioned 
stimulus, or, more generally, the 
arrangement of following the condi- 
tioned stimulus and response by re- 
ward or punishment, or some sub- 
stitute for these. 2. The process 
which increases the strength of con- 
ditioning as a consequence of these 
arrangements. (Distg. fr. facilita- 
tion; cf. derived reinforcement, 
heterogeneous reinforcement, 
homogeneous reinforcement, 
partial reinforcement) 


respondent (Skinner). That 
form of response which is elicited by 
a stimulus, and hence to be distin- 
guished from an operant. (Cf. op- 
erant) 


response. A stimulus-occasioned 
act. A general term for acts corre- 
lated with stimuli, whether the cor- 
relation is untrained or the result of 
training. (Cf. reflex) 
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response generalization. The 
behavioral fact that following the 
conditioning of one response, other 
responses may be elicited by the 
same conditioned stimulus although 
there has been no specific training 
to establish a relationship between 
this stimulus and these responses. 
(Syn. equivalence of responses; 
transfer; distg. fr. sensory gen- 
eralization) 


response tendency. A tendency 
to respond which is inferred from 
other responses. The tendency may 
be expressed in units of behavior if 
it is inferred from measured behav- 
ior. 


retroactive inhibition. A prin- 
ciple coming chiefly from experi- 
ments on verbal memorization and 
retention which accounts for forget- 
ting as the result of disruption or in- 
terference by new learning interpo- 
lated between memorization and 
recall. (Cf. forgetting) 


secondary conditioned re- 
sponse (Pavlov), see higher-or- 
der conditioned response 


Secondary extinction (Pav- 
lov). The extinction of other condi- 
tioned responses which results when 
any one conditioned response under- 
goes extinction. (Syn. generalized 
extinction; cf. generalization) 


secondary reward. A stimulus 
which derives its reinforcing value 
from prior conditioning in which it 
has been associated with a primary 
reward. (Cf. derived reinforce- 
ment; sub-goal; token) 
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secondary reward training. 
That arrangement of an instrumen- 
tal reward experiment in which the 
reinforcement is by way of a second- 
ary reward. (Cf. instrumental re- 
ward training) 


semi-voluntary response. A 
response which lies midway on the 
continuum between involuntary and 
voluntary responses in that it is de- 
termined both by the experimental 
stimulus and by instructions. Many 
conditioned responses are of this na- 
ture. 


sensitization (Wendt). The in- 
crease in the strength of a reflex 
originally evoked by a conditioned 
stimulus through its conjunction 
with an unconditioned stimulus and 
response. Differs from conventional 
conditioning in that the response 
which is strengthened is appropriate 
to the conditioned stimulus, not to 
the unconditioned stimulus. (Syn. 
alpha conditioning [Hull]; cf. 
pseudo-conditioning [Grether]) 


sensory generalization, see 
generalization 


set. A selective predisposition to 
respond to a stimulus in a prescribed 
manner which is established by 1n- 
structions or by some substitute for 
instructions. (Cf. expectancy) 


sign learning (Tolman). The 
formation of an expectation that 4 
first stimulus-object is going to be 
followed by a second stimulus-object, 
as testified by behavior preparatory 
to the second stimulus-object. (Sy2- 
sign-Gestalt learning) 


GLOSSARY 


simultaneous conditioning. 
1. Preferably, the procedure of con- 
ditioning in which the onset of con- 
ditioned and unconditioned stimuli 
is coincident. This is ordinarily an 
unfavorable relationship for condi- 
tioning. 2. The procedure in which 
the conditioned stimulus is presented 
from a fraction of a second to 5 sec- 
onds before the unconditioned stim- 
ulus, and continues until reinforce- 
ment occurs (Pavlov). 


spatial generalization. That 
form of sensory generalization in 
which the gradient of generalization 
of conditioning or of extinction is 
expressed as a function of the dis- 
tance between stimuli, usually on the 
body surface. (Cf. generalization; 
irradiation) 


spontaneous movement, see 
random movement 


spontaneous recovery (Pav- 
lov). The return in strength of a 
conditioned response, whether par- 
tial or complete, brought about by 
lapse of time following its diminu- 
tion by extinction. 


spread of effect (Thorndike). 
Rewarding a particular act (con- 
nection) leads not only to the 
strengthening of this act but to the 
strengthening of neighboring acts 
both preceding and following the 
rewarded act, the influence dimin- 
ishing as the acts are more remote 
from the rewarded one. (Cf. effect 
principle; gradient of reward) 


stimulus. An external or internal 
object or event which occasions an 
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alteration in the behavior of the 
organism. 


strength of conditioning. The 
extent to which a conditioned stim- 
ulus tends to evoke a given condi- 
tioned response, as inferred from the 
strength of response to the condi- 
tioned stimulus. Strength of re- 
sponse is not, however, a direct 
measure of strength of condition- 
ing, for in inferring strength of con- 
ditioning due regard must be paid 
to prior and subsequent responses. 
Thus, of two responses equal in 
magnitude, one may represent an 
inhibited response, another may be 
facilitated, and strength of condi- 
tioning may differ accordingly. Sim- 
jlarly, responses unequal in magni- 
tude may represent equal strengths 
of conditioning, if the conditioned 
stimuli differ in intensity. (Syn. 
associative strength; distg. fr. 
strength of response) 


strength of response. Any de- 
scriptive characteristic of single re- 
sponses (such as latency or ampli- 
tude) or of groups of responses (such 
as percentage frequency or rate) 
which may be expressed quantita- 
tively and hence enter as data with 
respect to magnitude or vigor of 
response. Which measures of strength 
of response are most serviceable 
must be determined empirically. 
(Distg. fr. strength of condition- 


ing) 


sub-goal. A stimulus regularly 
reacted to in the series of events 
leading up to the goal, so that this 
stimulus derives reinforcing value 
from its association with the goal. 
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(Cf. derived reinforcement; 
secondary reward; token) 


substitution principle. The 
principle explanatory of condition- 
ing which states that a conditioned 
stimulus acting at the time of an 
unconditioned response tends on its 
recurrence to evoke that response, 
and hence may be said to substitute 
for the unconditioned stimulus. (Cf. 
effect principle; expectancy 
principle) 


temporal conditioned re- 
sponse. When an unconditioned 
stimulus is presented at regular in- 
tervals, a conditioned response may 
appear at or near the usual time of 
appearance of the unconditioned 
stimulus, even if the unconditioned 
stimulus is omitted. Such response 
is known as a temporal conditioned 
response. 


temporal gradient in avoid- 
ance training, see gradients of 
reinforcement 


temporal gradient in class- 
ical conditioning, see gradients 
of reinforcement 


temporal gradient in delayed 
reaction experiments, see gra- 
dients of reinforcement 


token. A secondary reward ckar- 
acterized by the fact that it must 
be manipulated in a prescribed man- 
ner in order to produce the primary 
reward. (Cf. secondary reward) 
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trace conditioning (Pavlov). 
The conditioned stimulus is per- 
mitted to act and then is removed 
before the onset of the uncondi- 
tioned stimulus. Trace conditioned 
responses are more difficult to estab- 
lish than delayed conditioned re- 
sponses. In short-trace condi- 
tioning the conditioned stimulus is 
terminated a few seconds before the 
unconditioned stimulus is presented; 
in long-trace conditioning the 
conditioned stimulus is terminated a 
minute or more before the uncondi- 
tioned stimulus begins to act. 
(Distg. fr. delayed conditioning) 


trial-and-error learning 
(Thorndike). The mode of learning 
in which the learner tries various 
movements in its repertory, appar- 
ently in a somewhat random manner 
and without explicit recognition of 
the connection between the move- 
ment and the resolution of the prob- 
lem situation. Tentative movements 
which succeed are more frequently 
repeated in subsequent trials, and 
those which fail gradually disap- 
pear. (Syn. selecting and con- 
necting [Thorndike]; contr. W. 
associative shifting [Thorn- 
dike] ) 


types of nervous system 
(Pavlov). A classification of indi- 
vidual differences in conditioning, 
on the basis of the relative suscep- 
tibility to excitation and inhibition. 
The final types, with their corre- 
spondence to the ancient Greek clas- 
sifications of temperaments, are as 
follows: (1) excitable type (chol- 
eric), (2) central type, with two 


GLOSSARY 


sub-types, lively type (sanguine), 
and stolid type (phlegmatic), (3) 
inhibitable type (melancholic). 


ultra-paradoxical phase (Pav- 
lov). A transitory state of the cor- 
tical processes sometimes occurring 
before complete inhibition, in which 
only inhibitory agents have a posi- 
tive effect. (Cf. phase of equaliza- 
tion; paradoxical phase) 


unconditioned stimulus (Pav- 
lov). That experimental stimulus in 
a classical conditioning experiment 
which evokes the unconditioned re- 
sponse, and hence serves as the re- 


353 


inforcing agent. (Syn. uncondi- 
tional stimulus; conditioning 
stimulus [Razran]; original 
stimulus [Guthrie]) 


voluntary response. A re- 
sponse largely under the control of 
instructions or substitutes for in- 
structions, so that the experimental 
stimulus acts merely as an occasion 
for the response, and is of much less 
importance than instructions in de- 
termining whether or not the re- 
sponse occurs, or the form of the 
response when it occurs. (Cf. in- 
voluntary response; semi-vol- 
untary response) 
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221 
rôle of kinesthesis, 227 
Measures of response, 136 
reliabilities of, 137; references, 
154f. 
Mediated generalization, 
253 
defined, 344 
Mediated similarity, 201 
Memorization, rote, 221-226; refer- 
ences, 227 
Mental hygiene, conditioning in, 11, 
14, 25, 295 
Method of contrasts, 185, 343 


229-233, 
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Minimum latency of conditioned re- Neurosis, cont. 


sponses, 162; references, 174 
in relation to voluntary control, 
277 
Modifications not depending upon 
double stimulation, 41-43 
pseudo-conditioning, 41 
sensitization, 42 
Momentary rate, 137 
Morphine, 37, 323 
addiction, references, 307 
Morphological form, 301, 308 
Mosaic of functions, in cortex, 311, 
347 
Motivation, references, 155 
and substitution principle, 79 
Motor pathway, direct stimulation 
of, 322 
Multiple-path behavior, 242 


Narcolepsy, references, 307 
Negative adaptation, 105 
See also Adaptation 
Negative conditioned stimulus, de- 
fined, 347 
Negative induction, 310, 347 
Neural theory, in relation to condi- 
tioning theory, 336 
of effect principle, 81, 332 
of synaptic modification, 326f. 
Neurasthenia, 304 
Neurons, activity in closed chains, 
329f., 331 
excitation of, 327, 329 
many involved in conditioning, 
332f. Bette 
Neurophysiology of conditioning, 
309-339 
See also Cortex 
Neurosis, experimental, 279, 280- 
287; references, 306f. 
and psychoanalysis, 290-293, 307 


circumstances favoring, 280, 307 
in birds, reference, 306 
in cats, references, 306 
in children, 286f., 306f. 
in dogs, 279-281, 306 
in pigs, 283f., 306 
in rats, 284-286, 306 
in sheep, 281-283, 306 
relationship to human neurosis, 
280, 289f. 
theories of, 287-293, 304 
Neurosis, human, 280, 289f., 304, 
308 
Non-neural conditioning, 322f., 338 


Objectivism in psychology, 6, 9 
Sechenov’s, 23 
Obsessional neurosis, 304, 308 
Olfaction, references, 205 
Ontogenetic differences, in condi- 
tioning, 140, 155 
Operant behavior, 66, 72, 347 
and voluntary action, 277 
Organisms conditioned, 29, 49 
Original response to conditioned 
stimulus, I1I-114 
Original stimulus for conditioned re- 
sponse, 69 


Pancreatic secretion, references, 338 

Paradoxical phase, 288, 347 

Paralysis, during attempted condi- 
tioning, 323f. 

Paralysis, progressive, references, 308 

Paranoia, 304, 308 

Parathyroid, references, 156 

Partial identity as a form of similar- 
ity, 201 

Partial reinforcement, 148-153, 347 

in discrimination learning, 193f. 
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Pathways, functional, in condition- 
ing, 317-326 
association, 324-326 
of conditioned stimulus, 319-321 
of unconditioned response, 322- 
324 
of unconditioned stimulus, 321f. 
Patterns, of lights, 198 
of simultaneous stimuli, 198, 206 
of successive stimuli, -199, 206 
of tones, 198, 200 
Perceptual learning, 197, 204 
Periodic reconditioning, 150f., 166, 
347 
Personality, 279-308 
measures of, correlated with con- 
ditioning, 303, 308 
Phase of equalization, 288, 348 
Phylogenetic differences in condi- 
tioning, 139; references, 
155 
Positive induction, 310, 348 
Postulates, status of, 207f. 
Preferences, conditioned, 296; refer- 
ences, 307 
Preparatory responses, 348 
in classical conditioning, 39 
Pre-solution period in discrimination 
learning, 189f., 204 
Primary punishment, defined, 348 
Primary reward, defined, 348 
Probability theory of reinforcement, 
1938. 
Problem-solving, 228-254 
capacity, 246-248 
Proprioceptive stimuli, as integra- 
tors, 232, 240 
Pseudo-conditioning, 41, 49, 348 
and backward conditioning, 174 
Pseudo-reflex, 72, 348 
Psychoanalytic doctrines, and con- 
ditioning, 290-293; references, 
307 
Psychogenic depression, references, 
307 
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Psychotherapy, 296-299; references, 
307 
Punishment, inhibition by, 125 
relative to reward, 84 
Pupillary response, conditioned, 
2731.; references, 278 
Pure stimulus act, 230 
defined, 348 


Purposivism, IOI 


Random movements, 348 
in instrumental training, 66 
Rate of elicitation, and condition- 
ing, 148f. 
and extinction, 118, 154; refer- 
ences, 131 
Ratios of excitation, as favorable for 
conditioning, 144 
Reaction-produced stimuli, as con- 
ditioned stimuli, 171f., 202, 
226 
in serial learning, 211 
Reactions used as unconditioned re- 
sponses, 33 
Reasoning experiments, 237-241 
Recency principle, 91f., 102, 349 
Reconditioning, 349 
Recording equipment, references, 49 
Records of conditioned responses, 
30, 32, 38, 231 
Redintegration, 24, 37 
defined, 349 
Reflex activity, 336, 349 
compared with voluntary, 257f. 
Reflex reserve, 137, 146, 155, 166, 
349 
Reflexology, 9 
Regression to earlier habits, 114f. 
Reinforcement, 45, 75-103, 349 
kinds of, in different types of ex- 
periment, 69-71 
of serial reactions, 212 
principles of, 75-103 
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Reinforcement, principles of, cont. 
generality of each principle, 
90-97 
present status of problem, 97- 
100 
related to Skinner’s Type S 
and Type R, 100 

probability theory of, 193f. 

Reinforcement at a fixed ratio, 152, 
348 

Reliabilities of measures of response, 
137; references, 154f. 

Repeated conditioning and extinc- 
tion, 153-155 

Repetition, as affecting strength of 
conditioning, 145-154 

Repression, explanation of, 291, 293 

Respondent behavior, 66, 72, 349 

Response, defined, 349 

tendency, defined, 350 

Response generalization, 176, 184, 
350 

Response to relationships, 190-192 

Retention, of conditioning and ex- 
tinction, 127; references, 129 

Retroactive inhibition, 127, 350 

Retroflex, theory of, 82, 332 

Reward training, 52-56, 346; refer- 
ences, 73 

Rote memorization, 221-226; refer- 
ences, 227 


Schizophrenia, 304f.; references, 308 
Secondary extinction, 131, 350 
Secondary reward training, 62-65, 
244, 350; references, 74 
Selection and elimination by prin- 
ciple of effect, 84 
Self-initiated activity, 256, 276, 278 
Semi-voluntary conditioned re- 
sponses, 261-267, 350 
Senile psychosis, reference, 308 


Sensitization, 42, 49, 35° | 
and backward conditioning, 174 
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Sensory capacities of animals, refer- 
ences, 204 
Sensory conditioning, 50 
Sensory generalization, 176-185 
See also Generalization 
Sensory similarity, 201 
Serial learning, 207-227 
maze, 216-221; references, 227 
verbal, 221-226; references, 227 
Serial reactions, condensation of, 214 
reinforcement of, 212 
Set, 102f., 263-267, 350 
in mediated stimulus equivalence, 
232 
Sexual aberrations, references, 307 
Shock, various experimental usages, 
74 
Sign learning, 88, 102, 204, 350 
explanation in terms of condi- 
tioning, 233, 236 
Sign-urge theory, 90 
Similarity, defined, 200 
and difference, 197-202, 205 
See also Equivalence, Generaliza- 
tion 
Simultaneous conditioning, 44, 351 
Skill, organization of, 251 
Sleep, references, 131, 156 
as generalized inhibition, 117, 
288, 311 
Social factors in conditioning experi- 
ments, 251 
Source of conditioned response in 
instrumental training, 66-68 
Spatial generalization, 179f., 351 
Spatial gradient of avoidance re- 
sponses, 170 
Spinal cord, isolated, conditioning in, 
3371. 
Spontaneous movements, see Oper- 
ant behavior 
Spontaneous recovery, 45, 132, 351; 
references, 133 
Spread of effect, 96, 167f., 351 
as related to goal-gradient, 174 
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Static and kinetic reactions, refer- 
ences, 205 
Stimulus, usage of, 73 
Stimulus trace, 1711. 
Strength of conditioning, 135-157, 
351 
and generalization, 180-182 
and strength of reinforcement, 
143!. 
intercorrelation of measures of, 
138, 155 
Strength of reinforcement, 143f. 
Strength of response, 135-139, 351 
as criterion of strength of con- 
ditioning, 136 
Strychnine, 320f. 
Stuttering, reference, 307 
Sub-cortical conditioning, 314f. 
Sub-goal, 64, 351 
Subliminal stimuli as conditioned 
stimuli, references, 205 
Substitution, as a principle of rein- 
forcement, 76-80, 90-92, IOI, 


352 
compared with effect principle, 
8sf., 9I, IOI 


generality of, 90-92 
Guthrie’s interpretation, 76-78, 
91-93 
in serial learning, 212 
Pavlov’s interpretation, 76-78 
Suggestibility, correlated with con- 
ditioning, 303, 308 
Synaptic modification in learning, 
theories of, 326-334; refer- 
ences, 338f. 
anatomical growth of dendrites 
and axons, 327f. 
bioelectric fields, 330f. 
changes in physico-chemical 
properties of nerve cell, 328f. 
continued activity in closed neu- 
ral chains, 329-331 
objections to, in learning theory, 
3338. 
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Technique, references, 49 
buried coil, for faradic stimula- 
tion of nervous system, 319f. 
Temperature sensitivity, reference, 
205 
Temporal conditioned response, 45, 
352 
Temporal gradient, in avoidance 
learning, 164f. 
in classical conditioning, 159-163, 
174 
in delayed reaction experiment, 
168f. 

Temporal relationships within con- 
ditioning, 44, 158-175 
Tension reduction, as characterizing 

reinforcement, 82f. 
in avoidance learning, 245f. 
in explanation of amnesia, 292f. 
Terminology, 48 


Threshold measurement, 195-197, 
204f. 
conditioning compared with 


other methods, 195 
Thyroid, references, 156 
Time discrimination experiments, as 
a form of delayed reward, 174 
Token rewards, 62, 352; references, 
74 
Trace conditioned response, 44, 352 
difficulty in establishment rela- 
tive to delayed, 163 
in serial verbal learning, 222, 225 
latency, 163, 174 
wide generalization of, 202 
Transfer of conditioning, 206, 243f. 
Transposition, 190, 204 
Trial-and-error learning, 53, 252, 
352 as 
Types of conditioning, classical con- 
ditioning, 26-50 
instrumental conditioning, 51-74 
Skinners Type S and Type R, 
72, 100 
various alternatives, 54, 71 
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Types of nervous system 
according to Pavlov, 300f., 307, 
352 
according to others, references, 
308 


Ultra-paradoxical phase, 288, 353 
Unconditioned response, compared 
with conditioned, 36, 49 

table of reactions used, 33 
Unconditioned stimulus, defined, 353 


Vasomotor responses, conditioned, 


274, 278 
Vibratory sense, reference, 205 


Vicarious functioning of cortex, 315 


Vision, 196; references, 205 
Voluntary action, 255-278, 353 
compared with reflex, 2571. 
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Voluntary conditioned responses, 
267-270; references, 277 
Voluntary control, acquisition of, 

273-278 

degree of, general, 255f., 277 

degree of, over conditioned re- 
sponses, 259-272, 277 

of pupillary response, 273f., refer- 
ences, 278 

over autonomic activities, 274 


Word-number experiment, as test of 
gradient of reward, 96 
illustrating spread of effect, 167 
Work-decrement, 106 


X-ray radiation of brain, 1 56 
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